Improving spectral efficiency under a certain energy limitation is an important design metric for future wireless communications as a response to the growing transmission demand of wireless devices. In order to improve spectral efficiency for communication systems without increasing energy consumption, this paper considers a non-orthogonal multiple access (NOMA)-based cognitive radio network, with the assistance of a wireless-powered relay station (RS), and then analyzes the system outage performance under amplified-and-forward (AF) and decoded-and-forward (DF) cooperative transmission modes. Specifically, the base station (BS) has the opportunity to cooperate by transmitting information through the RS, depending on whether the RS can harvest sufficient RF energy for cooperative transmission. That is to say, when the energy stored by the RS is sufficient for cooperative transmission, the RS will assist the BS to forward information; otherwise, the BS will send information through direct links, while the RS converts the radio frequency (RF) signals sent by the BS into energy for future transmission. Moreover, the transmission power required by the RS for cooperative transmission is usually relatively large, while the amount of harvested energy by the RS in a transmission slot is usually low, so it takes several consecutive time slots to accumulate enough transmission energy. To this end, we utilize a discrete-time Markov chain to describe the processes of charging and discharging of the RS. Subsequently, we derive the closed-form outage probabilities of both the primary and secondary systems for the considered system in AF and DF modes through mathematical analysis, and verify the accuracy of the analyses through Monte Carlo simulation. The simulation results show that the two proposed cooperative transmission schemes with AF and DF relaying techniques outperform both direct transmission and other similar schemes in both the primary and secondary system, while the DF scheme can provide better performance than the AF scheme within the range of setting values.