Parallel Disassembly Sequence Planning (DSP) involves determining the disassembly sequence by disassembling multiple parts in parallel at certain disassembly levels of a product. Previous studies only used the precedence graph to represent product assembly and determine part retrieval order. This does not guarantee that all parts are disassembled in a safe and stable manner, underscoring the importance of stability constraints in disassembly. Introducing stability constraints makes it difficult to completely disassemble the product with lower disassembly levels while minimizing direction changes due to the combinatorial explosion of the problem. This article proposes a two-stage approach to parallel DSP using Reinforcement Learning (RL). In the first stage, the mathematical model is introduced followed by a stability-integrated precedence graph for the product to simplify the problem. Then, obtain various part combinations of the product. In the next stage, a model-free RL formulation of the problem and an Adaptive Simulated Annealing Linear Decay-based Q-learning (ASALD-Q) algorithm is proposed to adjust the ASALD parameters dynamically and avoid local optima, resulting in faster convergence. Adoption of sparse Q-table representation makes the ASALD-Q learning algorithm generalize well to complex products. The proposed algorithm is applied to four products of varying complexity, including the product reported in the literature, and the results are compared with another model-free RL algorithm with two exploration techniques. The results demonstrate the efficacy and relevance of the proposed approach, mainly when dealing with large-scale products.