The Limitations of Using the Pittsburgh Sleep Quality Index to Assess Athletes’ Sleep Quality: Evidence from Reliability and Validity in Chinese Professional Athletes

Introduction

The significance of sleep in relation to athlete health and performance has garnered increased attention. Research on athlete sleep has expanded quickly in recent years, influencing even daily management and training plans.^1–5

Within the realm of athlete sleep research, subjective sleep rating scales, known for their simplicity, quick administration, and comprehensive nature compared to specific objective indicators, have found widespread application across various research themes.^6–10 Among these scales, the Pittsburgh Sleep Quality Index (PSQI) is one of the most extensively used tools.

The PSQI was introduced in the 1980s as a subjective assessment scale tailored to clinical needs.¹¹ In the decades since its inception, the PSQI has played a pivotal role in both clinical applications and scientific research, even serving as a benchmark for other sleep measurement tools in terms of convergent validity.¹² Concerning athlete sleep, whether in comparing and analyzing sleep quality issues or intervening in sleep quality, the PSQI has emerged as the primary subjective assessment tool.^1,13 It can be argued that, concerning subjective reports of sleep quality, the current understanding and application of athlete sleep may be grounded in using the PSQI.

However, there currently needs to be more research on the structural validity of the PSQI factor structure among athletes, as well as a relative absence of systematic examination regarding its reliability and convergent validity. This absence of evidence presents a challenge to effectively utilizing the scale to study athletes’ sleep patterns. Indeed, researchers have already presented varying views on the utility of the PSQI in athletes, with studies suggesting that it may not accurately reflect existing sleep issues.^1,14,15

The structural validity of the PSQI faces immediate challenges. Its unidimensional structure is a subject of considerable debate. Studies in non-athlete populations have frequently found suboptimal fit for the single-factor structure, with many not supporting this configuration.^12,16,17 Increasingly, researchers have been analyzing PSQI items separately as dependent variables beyond the total score,^1,8,14,18,19 uncovering group differences not evident in the overall score.¹⁴

Regarding the factor structure of the PSQI, two-factor structures and three-factor structures with more optimal model fitness have been successively discovered across different languages and populations.¹⁶ However, the optimal structure varies among different populations. For instance, a two-factor structure was found to be superior among female workers in New Zealand during lockdown,²⁰ mirroring findings observed in Filipino Domestic Workers,²¹ and among university students in Southeast Asia, Africa, and South America.²² Similarly, comparable factor structures have been observed among university students in China.²³ However, a three-factor structure was found to be more optimal among elderly Americans, with similar results validated among medical students in Iran.^24,25 In comparison to other groups such as students, older people, and other occupational groups, evidence remains scarce regarding the factor structure of the PSQI among professional athletes, including Chinese athletes, and the structural validity based on theories such as the unidimensional model.

Structural validity reflects the rationality of the PSQI factor structure. Yet, when assessing athletes’ sleep quality using the PSQI, one must also consider the stability and consistency of measurement results. Regarding the reliability of the PSQI, there is currently a lack of reliable Conclusions regarding test-retest reliability, and a reliable retest interval cannot be determined.¹⁷ Good reliability forms the basis of good validity. Without reliable reliability characteristics, the Convergent Validity of using the PSQI in athletes, ie, whether it can effectively measure corresponding sleep quality, remains to be supplemented with evidence. Combined with the controversies surrounding structural validity, the differing factor structures across different populations also suggest that the reliability and validity testing of subdimensions in athletes are still incomplete.

It is noteworthy that when considering the test-retest reliability and convergent validity of the PSQI, special attention should be paid to the time frame within which the PSQI is used. When the PSQI was developed, its measurement time frame was set at one month based on its clinical use and symptoms for diagnosing insomnia.¹¹ The commonly used course criteria for insomnia may have been the initial reason for this timeframe.¹⁷

Thus, when describing non-clinical patients, especially when sleep conditions are related to training and competition factors, does such a time frame limit the effectiveness of the PSQI? Some researchers have proactively modified the PSQI time frame, requiring participants to respond based on their recent 1–2 weeks’ condition, to demonstrate the short-term impact of specific external factors on sleep quality.^26,27 Although this approach has not been tested, it suggests the need to consider the selection and application of time frames when using the PSQI. For athletes, the factors they face, such as training and competition, mostly manifest as changes within a shorter period. Can the longer time frame used according to clinical diagnostic criteria still yield reliable measurement results for such a population? Evidence in this regard is also lacking, but this demand for non-clinical practice must be considered in testing among athlete populations.

To the best of our knowledge, this study will be the first to systematically examine the reliability and validity of the Chinese version of the PSQI among professional athletes in China. Specifically, exploratory factor analysis (EFA) will first be used among athletes to conduct data-driven analyses of the PSQI factor structure, thereby preliminarily exploring the characteristics of the PSQI factor structure among athletes. Subsequently, confirmatory factor analysis (CFA) will be employed to test the fitness of the factor structure obtained from EFA, as well as the fitness of single-factor, two-factor, and three-factor structures proposed in previous studies, to examine and compare the structural validity of the PSQI factor structure.^23,28 Next, the test-retest reliability of the PSQI at different time intervals will be analyzed. The timeframe of the PSQI will then be changed to one week, and the test-retest reliability and convergent validity of the PSQI will be tested again. Finally, a one-month follow-up will be conducted to analyze the relationship between the results obtained within a one-week timeframe per week and the results obtained within a two-week and one-month timeframe, providing evidence for the convergent validity of the PSQI under a longer timeframe. Based on the results of these sections, this study will fill the gaps in the reliability and validity of the PSQI among athletes and provide constructive suggestions for the more effective use of the PSQI among athletes in China to optimize training effectiveness and safeguard their well-being.

Material and Methods

Participants

A total of 581 Chinese professional athletes participated in this study. These athletes were recruited from various professional sports teams managed by the Shanghai Sports Administration, spanning a diverse range of disciplines including badminton (n = 72), shooting (n = 65), handball (n = 58), martial arts (n = 45), fencing (n = 44), archery (n = 42), boxing (n = 37), field hockey (n = 35), gymnastics (n = 33), modern pentathlon (n = 31), judo (n = 25), table tennis (n = 23), basketball (n = 22), taekwondo (n = 18), chess (n = 15; encompassing chess, international checkers, Chinese chess), cycling (n = 9), athletics (n = 5), baseball (n = 1), and karate (n = 1). In addition to volunteering to participate, these athletes obtained permission from their coaches and management personnel. It is noteworthy that, due to the athlete development pathway and daily management system of sports teams in Chinese competitive sports, these athletes were mainly in the age range equivalent to secondary school to university, unmarried for the majority, and holding positions equivalent to full-time government-employed professionals with a certain fixed income. Furthermore, while most received education from secondary to graduate levels within their training base, their daily routines were predominantly centered around rigorous training regimens, setting them apart significantly from student-athletes.

The athletes in this study accounted for more than half of the total number of athletes in professional sports teams operated and managed by the Shanghai Sports Administration throughout the city. They undergo long-term specialized training at training bases and are eligible to represent Shanghai in the highest-level competitions nationwide. Some athletes have been selected for the Chinese national team, and a few have represented China in high-level international competitions during their tenure with the national team. Within this study, a subset of athletes participated in various reliability and validity assessments, resulting in a cumulative sample size of 909 individuals distributed across eight samples. Participation was limited to once per sample for athletes included in multiple samples.

(1) A total of 489 athletes participated in the analysis of the factor structure and the test of the structural validity of the PSQI. These athletes represented various sports, including shooting (n = 61), handball (n = 55), badminton (n = 55), fencing (n = 38), martial arts (n = 37), field hockey (n = 35), gymnastics (n = 33), boxing (n = 31), modern pentathlon (n = 29), archery (n = 24), taekwondo (n = 18), table tennis (n = 16), basketball (n = 13), chess (n = 15), cycling (n = 12), judo (n = 10), athletics (n = 5), baseball (n = 1), as well as karate (n = 1). Based on their sports and gender, they were stratified and randomly assigned to two matched samples for exploratory factor analysis (EFA - Sample 1) and confirmatory factor analysis (CFA - Sample 2). Sample 1 comprised 244 athletes, with 123 males and 121 females, with a mean age of 20.05 years (SD = 4.81). Sample 2 included 245 athletes, with 112 males and 133 females, with a mean age of 20.15 years (SD = 4.38).

(2) Sample 3 was used to examine the 2-week test-retest reliability of the PSQI, with 115 athletes (65 males, 50 females), averaging 19.23 years (SD = 3.78). Their sports included handball (n = 27), badminton (n = 26), fencing (n = 24), judo (n = 22), modern pentathlon (n = 10), and table tennis (n = 6). Sample 4 was employed for testing the 1-week test-retest reliability of the PSQI, involving 125 athletes (69 males, 56 females), averaging 19.62 years (SD = 3.99). Their sports included fencing (n = 27), handball (n = 27), boxing (n = 24), badminton (n = 18), martial arts (n = 15), basketball (n = 5), table tennis (n = 5), and judo (n = 4). Sample 5 was designated for evaluating the 2–3 days test-retest reliability of the PSQI, with 37 athletes (23 males, 14 females), averaging 23.03 years (SD = 6.67). Their sports included shooting (n = 19) and archery (n = 18).

(3) Sample 6 was used to assess the test-retest reliability of the PSQI within a 1-week timeframe, with 57 athletes (21 males, 36 females), averaging 19.21 years (SD = 3.04). Their sports included archery (n = 19), badminton (n = 16), basketball (n = 15), and shooting (n = 7). Sample 7 was designated for evaluating the convergent validity of the PSQI with a 1-week timeframe, involving 61 athletes (39 males, 22 females), averaging 21.15 years (SD = 3.65). Their sports included badminton (n = 17), fencing (n = 13), martial arts (n = 11), modern pentathlon (n = 10), handball (n = 8), and table tennis (n = 2).

(4) Thirty-six athletes who were confined to the training base for at least one month without leaving, while engaging in regular training and with no recent competition tasks, were recruited. Eleven athletes were excluded from the initial 36 due to withdrawals or missing at least one test because of unforeseen decisions, such as participating in external competitions or exams within the four weeks. The remaining 25 athletes constituted Sample 8, with 7 males and 18 females. Their sports included modern pentathlon (n = 11), basketball (n = 11), and archery (n = 3), averaging 20.44 years (SD = 3.47).

The study protocol received approval from the Ethics Committee of the Shanghai Research Institute of Sports Science (Shanghai Anti-doping Agency) (Ethics Approval Number LLSC 20220005). It adhered to the ethical principles outlined in the latest version of the Declaration of Helsinki. All Participants provided written consent before participating in the study.

Measures

The Chinese version of the Pittsburgh Sleep Quality Index (PSQI)

The PSQI, comprising 18 self-assessment items, assesses seven components, including subjective sleep quality, sleep latency, sleep duration, sleep efficiency, sleep disturbance, sleep medication, and daytime dysfunction.¹¹ The Chinese version of the PSQI has been widely used in China, and its reliability and validity have been tested in populations other than athletes.^23,28,29

For Samples 1–5, athletes responded based on their recent one-month status. For Samples 6–7, athletes reported their state within the past week. In the case of Sample 8, athletes completed PSQI every Monday based on the responses from the preceding week. On the third Tuesday, an assessment was conducted based on responses from the prior two weeks. Similarly, on the fifth Tuesday, an assessment was made based on responses from the past month.

The Chinese version of the Insomnia Severity Index (ISI)

The ISI comprises seven items, each scored on a 0–4 scale, with higher scores indicating more severe insomnia symptoms.³⁰ In this study, the wording of ISI instructions and items related to the timeframe was modified to reflect the recent one-week period, while the remaining content remained unchanged.³¹ In the current study, the ISI was used to test the convergent validity of the PSQI within a one-week timeframe. The reliability and validity of the Chinese version of the ISI have been tested and widely applied in various populations in China.^32,33

The Chinese version of the Athlete Sleep Screening Questionnaire (ASSQ)

The ASSQ is a scale explicitly developed for screening sleep issues and guiding interventions in athletes. The Sleep Difficulty Score (SDS), comprising five items, reflects the severity of sleep difficulties, with higher scores indicating greater difficulty.¹⁵ In the current study, the ASSQ-SDS was used to test the convergent validity of the PSQI within a one-week timeframe. The Chinese version of ASSQ has been revised and validated among Chinese athletes and applied in research involving Chinese athletes.^34,35

The Chinese version of the Consensus Sleep Diary-Core (CSD-C)

The Consensus Sleep Diary is a standardized tool introduced by sleep researchers for using sleep diaries in insomnia research.³⁶ In this study, five indicators—Latency, Duration (calculated based on sleep onset and final wake-up time), Total Sleep Time (TST, sleep duration minus sleep latency and wake after sleep onset time), Wake After Sleep Onset (WASO), and Perceived Sleep Quality—were utilized. The means of these indicators over one week served as the subjective record of weekly sleep quality, and they were used to test the convergent validity in the one-month tracking. The items in the Chinese version of CSD-C have been used in research on Chinese athletes’ athletic performance and sleep quality, demonstrating good effectiveness.³⁷

Actigraphy

The Actigraph wGT3X-BT (ActiGraph, LLC) recorded athletes’ physical activity during sleep for three consecutive training days within a week (from Tuesday to Thursday, or Wednesday to Friday). A minimum of two nights of sleep per week was considered valid data (one week for Sample 7 and four weeks for Sample 8). Utilizing ActiLife v6.13.4, in conjunction with athletes’ recorded sleep onset and wake-up times, sleep metrics, including Total Sleep Time (TST), Wake After Sleep Onset (WASO), and Sleep Efficiency, were computed. The weekly averages functioned as objective records of the sleep quality for that week. In this study, they were utilized to test the convergent validity of the PSQI within a one-week timeframe. They were also employed to assess the convergent validity in the one-month tracking.

Notably, the recommended minimum duration for continuous recording for clinical analysis is three consecutive days, ideally extending to 7–14 days.^38,39 However, since athletes are not primarily under clinical purview, their adherence to prolonged device usage for sleep monitoring may be limited. Additionally, coaches have expressed concerns about the potential adverse effects of prolonged watch-wearing on athletes’ training and mood. Consultations with coaches and referencing previous practices of employing three consecutive nights of recording among athletes, along with findings from studies on elderly populations indicating that a 3-day average reflects the average levels of 7 and 14 days of recording,^40,41 led to the decision to use the average results of three consecutive nights of sleep recording on training days to represent the sleep quality for the entire week in this study.

Procedures

For Samples 1–2, athletes, organized by sports teams, gathered at specified locations within designated time slots to individually complete the PSQI once online using their smartphones.

For Samples 3–5, athletes completed the PSQI for a second time after intervals of 2 weeks, 1 week, or 2–3 days, respectively, within a one-month timeframe. Athletes independently completed both tests on their smartphones within the specified time frame.

For Sample 6, athletes completed the PSQI on Monday and Wednesday of the same week, utilizing a one-week timeframe. Both assessments were conducted online.

For Sample 7, athletes were initially instructed on how to wear the Actigraph wGT3X-BT on the Monday of the first week. Subsequently, athletes were required to wear the device during the evenings of three consecutive training days within that week (with Monday night for adaptation) and record their sleep and wake-up times daily. In the following week, on Monday or Tuesday, athletes completed paper-based PSQI, ISI, and ASSQ at a designated location, all within a one-week timeframe.

For Sample 8, athletes, in addition to wearing the Actigraph wGT3X-BT, were also required to complete the CSD-C. Over four consecutive weeks, athletes completed the PSQI online every Monday based on their status of the previous week. Additionally, on the Tuesday of the third week, they completed the PSQI online with a 2-week timeframe, and on the Tuesday of the fifth week, they completed the PSQI online with a 1-month timeframe (Figure 1).

Figure 1 Procedure for measurements over one month.

Statistical Analysis

EFA was conducted using SPSS 26.0, and CFA was performed using Amos 20.0.

Before conducting EFA, the suitability of the data for conducting EFA was first examined using the Kaiser-Meyer-Olkin (KMO) measure and Bartlett’s test of sphericity. The recommended threshold for KMO values is equal to or greater than 0.7, but values between 0.5 and 0.7 are also considered acceptable.²⁵ For Bartlett’s test of sphericity, the critical value is a p-value less than 0.05. After confirming the suitability of the data for EFA, principal component analysis and maximum variance rotation were performed, with factors extracted based on eigenvalues exceeding 1.

Since athletes are non-clinical participants and adhere to rigorous individual medication management under anti-doping regulations, this study revealed that 90.4% of the total sample reported not using any sleep aids. In line with previous research highlighting low loadings of sleep medication and the practice of excluding such components in factor analysis,¹⁷ factors with low loadings were excluded for reanalysis.

For CFA, the structures identified in the EFA were compared with the single-factor, two-factor, and three-factor models tested in previous studies involving non-clinical samples in China.^23,28 The two-factor model comprised (a) Factor 1 - sleep duration, sleep efficiency, and Factor 2 - subjective sleep quality, sleep latency, sleep disturbance, and daytime dysfunction,²² and (b) Factor 1 - sleep duration, daytime dysfunction, subjective sleep quality, and Factor 2 - sleep medication, sleep disturbance, sleep efficiency, sleep latency.⁴² The three-factor model included Factor 1 - subjective sleep quality, sleep latency, and sleep medication; Factor 2 - sleep duration, sleep efficiency; and Factor 3 - sleep disturbance and daytime dysfunction.²⁴ The model’s fit was evaluated using the Root Mean Square Error of Approximation (RMSEA), Comparative Fit Index (CFI), Adjusted Goodness of Fit Index (AGFI), and Tucker–Lewis Index (TLI). Intraclass correlation coefficient (ICC) was employed to assess test-retest reliability, and Pearson correlation coefficients were used to examine criterion-related validity.

In the 1-month tracing data analysis, repeated Measures ANOVA was employed to test weekly PSQI scores, assessing the stability of athletes’ sleep quality. Pearson correlation coefficients examined the relationship between average weekly measurements, weekly measurements, and monthly measurements. Kendall’s τ coefficient analyzed the correlation between accelerometer-based objective measurements. Correlation differences were tested using cocor.⁴³ Athletes who did not wear Actigraph wGT3X-BT or failed to complete the scales were excluded from analyses involving these variables.

Results

EFA

Athletes in Sample 1 achieved an average score of 6.51 (SD = 3.07) on the total PSQI score across seven components and an average score of 6.37 (SD = 2.94) across six components (excluding sleep medicine).

Before conducting EFA, an evaluation of the data indicated a KMO value of 0.696, and Bartlett’s test of sphericity showed χ²(21) = 321.647, p < 0.001. These Results confirmed the data’s appropriateness for EFA. Two factors with eigenvalues greater than 1 were extracted, explaining 54.80% of the variance. Notably, component 6 (related to sleep medication) displayed low and multiple-loading, suggesting its possible elimination from the analysis.

After excluding the sleep medication component, EFA was conducted again. The analysis yielded a KMO value of 0.694, and Bartlett’s test of sphericity resulted in χ²(15) = 302.835, p < 0.001. These findings affirmed the suitability of the data for EFA. Subsequently, EFA was re-run. Two factors had an eigenvalue greater than 1, accounting for 62.71% of the variance. The six retained components had the same attribution on both factors as the seven components (Table 1).

Table 1 Factor Loadings of PSQI Among Athletes

CFA

Athletes in Sample 2 obtained an average score of 6.77 (SD = 3.43) on the PSQI (7 items) and an average score of 6.58 (SD = 3.22) on the PSQI (6 items). In the two-factor model derived from EFA, the average score for Sleep Quality was 5.27 (SD = 2.35), and the average score for Sleep Efficiency was 1.31 (SD = 1.43).

After removing the sleep drug component, the two-factor model obtained by EFA fitted well (Figure 2). The one-factor and the two-factor models obtained in the previous study were poorly fitted. The three-factor model fit was fair but not as good as the two-factor model obtained in the present study. If the sleep medication component is retained, the two-factor model obtained in this study fits reasonably and is similar to the three-factor model. The one-factor and two-factor models obtained in previous studies do not fit well (Table 2).

Table 2 Model Fit Indices of Different Models

Figure 2 CFA parameter estimation results for the two-factor model obtained based on EFA with the sleep medication component removed.

Retest Reliability (1-Month)

Table 3 presents the descriptive statistics for the scores of athletes completing the PSQI twice at different time intervals in Samples 3–5.

Table 3 Descriptive Statistics for PSQI Scores at Different Intervals for a Time Frame of 1 Month (M ± SD)

Both the 2-week and 1-week interval PSQI total scores demonstrated good retest reliability. The two-factor model’s sleep quality dimension also showed better retest reliability than the retest reliability of sleep efficiency. However, for the 2–3 days interval retest reliability, only the sleep quality dimension showed fair retest reliability, while the PSQI total score showed low but still significant retest reliability, and the result for sleep efficiency was poor (Table 4).

Table 4 ICC of PSQI at Different Intervals for a Time Frame of 1 Month

Retest Reliability and Convergent Validity (1-Week)

For the athletes in Sample 6, their average scores on the PSQI (7 items) were 5.91 (SD = 2.54) for the first completion and 5.84 (SD = 2.48) on the PSQI (6 items). In the two-factor model derived through EFA, the average score for Sleep Quality was 4.98 (SD = 2.12), and for Sleep Efficiency was 0.86 (SD = 0.97) during the first completion. For the second completion, the average scores were 5.47 (SD = 2.62) on the PSQI (7 items) and 5.37 (SD = 2.57) on the PSQI (6 items). In the EFA-derived two-factor model, Sleep Quality had an average score of 4.68 (SD = 2.21), and Sleep Efficiency scored 0.68 (SD = 1.00).

As for the athletes in Sample 7, the average scores on the PSQI (7 items) were 7.74 (SD = 3.13) and 7.51 (SD = 2.72) on the PSQI (6 items). In the two-factor model from EFA, the average score for Sleep Quality was 6.21 (SD = 1.78), and for Sleep Efficiency was 1.30 (SD = 1.49). The average score on the ISI was 9.34 (SD = 4.78), and for ASSQ-SDS, it was 6.90 (SD = 2.26). The average sleep efficiency recorded through actigraphy was 0.89 (SD = 0.04), with a mean total sleep time of 406.24 (SD = 42.99) and wake after sleep onset of 46.80 (SD = 18.55).

Athletes demonstrated good test-retest reliability with a two-day interval for the PSQI. Specifically, for the PSQI total score (7 items), ICC = 0.881, p < 0.001; for the PSQI total score (6 items), ICC = 0.877, p < 0.001; for the Sleep Quality score, ICC = 0.864, p < 0.001; and for the Sleep Efficiency score, ICC = 0.769, p < 0.001.

Within one week, the PSQI total score and the Sleep Quality dimension score of the PSQI two-factor model exhibited significantly positive correlations with subjective reports, as ISI and the SDS indicated. This demonstrates good convergent validity. However, the Sleep Efficiency dimension scores correlated positively with ISI and SDS of ASSQ, although the correlation coefficients did not reach a high level of association. Significantly, neither the total PSQI score nor the scores of its two dimensions exhibited any significant correlation with objective sleep quality indicators, including Sleep Efficiency, TST, and WASO (Table 5).

Table 5 Validity of the PSQI Within a 1-Week Timeframe (r)

One-Month Tracking

The results revealed significant variations in the PSQI total scores (7 items) over the four weeks, F(3, 72) = 5.928, p = 0.001, η_p² = 0.198. The total score in the first week was higher than in the second and third weeks (p_1-2 = 0.007, p_1-3 = 0.021), with no difference observed compared to the fourth week (p = 0.072). Similarly, a significant difference was observed for the PSQI total scores (6 items) over the four weeks, F(3,72) = 6.358, p = 0.001, η_p² = 0.209. The total score in the first week was higher than in the subsequent three weeks (p_1-2 = 0.008, p_1-3 = 0.022, p_1-4 = 0.039).

Concerning sleep quality scores, a significant difference was found, F(3,72) = 5.124, p = 0.003, η_p² = 0.176. The scores in the first week were higher than in the following two weeks (p_1-2 = 0.021, p_1-3 = 0.046), but no difference was noted compared to the fourth week (p = 0.051). In contrast, sleep efficiency scores did not reveal a significant difference, F(3,22) = 2.632, p = 0.075, η_p² = 0.264.

When comparing the results of the 2-week measurement with the mean derived from the two-week data, differences were observed in the PSQI total scores (7 items), F(1,21) = 4.852, p = 0.039, η_p² = 0.188, with the measurements being lower than the mean. Similarly, the PSQI total scores (6 items) showed differences, F(1,21) = 5.105, p = 0.035, η_p² = 0.196, with the measurements being lower than the mean. However, no differences were found in sleep quality, F(1,21) = 3.539, p = 0.074, η_p² = 0.144, and sleep efficiency, F(1,21) = 2.009, p = 0.171, η_p² = 0.087.

In the comparison between the one-month measurement results and the mean derived from four weeks of data, no differences were observed in the PSQI total scores (7 items), F(1,21) =1.074, p = 0.310, η_p² = 0.043, and the PSQI total scores (6 items), F(1,21) = 2.424, p = 0.133, η_p² = 0.092. Additionally, no differences were detected in sleep quality, F(1,21) = 0.349, p = 0.560, η_p² = 0.014, while a difference was observed in sleep efficiency, F(1,21) = 4.478, p = 0.045, η_p² = 0.157, with the measurements being lower than the mean. Statistical Descriptions of PSQI Scores at Different Time Points are presented in Table 6.

Table 6 Descriptive Statistics for PSQI Scores at Various Time Points (M ± SD)

Except for sleep efficiency dimension scores, the 2-week and 1-month PSQI total scores and sleep quality dimension scores exhibited relatively strong correlations with the results for each week. Notably, the closer the weekly measurement was to the week conducting the 2-week or 1-month interval test, the higher the correlation coefficients tended to be, or at least on par with those from the preceding week (see Table 7).

Table 7 Correlation Coefficients Between 2-Week and 1-Month PSQI Scores and Weekly PSQI Scores During the Period

Tests were conducted to examine the differences in correlation coefficients for PSQI total scores (7 items) and the sleep quality dimension that was most proximate to the long-term framework test week (the fourth week) compared to the coefficients for the preceding weeks. The results indicated that the one-month scores did not significantly differ from the correlations of the fourth week (Z = 0.920, p = 0.358), the third week (Z = 0.854, p = 0.393), or the first week (Z = 0.631, p = 0.528). However, the total scores for the two weeks showed a higher correlation with the second week than the first week (Z = 2.869, p = 0.041).

In the case of PSQI total scores (6 items), the results revealed that the one-month scores did not significantly differ from the correlations of the fourth week (Z = −0.572, p = 0.567) or the third week (Z = 1.432, p = 0.152), but they were higher than the correlations of the first week (Z = 2.010, p = 0.044). Additionally, the total scores for two weeks demonstrated a stronger correlation with the second week than the first week (Z = 2.643, p = 0.008).

Concerning sleep quality dimension scores, the one-month scores did not significantly differ from the correlations of the fourth week (Z = 0.496, p = 0.620). However, they were higher than the correlations of the second week (Z = 2.429, p = 0.014) and the first week (Z = 2.455, p = 0.014). The two-week scores showed a stronger correlation with the second week than the first week (Z = 2.335, p = 0.020).

The PSQI total and sleep quality dimension scores show significant correlations with average subjective quality and latency recorded subjectively (see Figure 3). However, there are no comparably stable correlations with indicators calculated from specific times in sleep diary records. Neither the PSQI total score nor the sleep quality dimension exhibits relatively stable correlations with objectively recorded sleep metrics. Nevertheless, the sleep efficiency dimension moderately correlates with objective sleep metrics in some individual weeks’ results.

Figure 3 Correlations between PSQI and validity variables.

Discussion

The PSQI revealed a two-factor model among athletes, aligning with structures observed in Southeast Asia, Africa, South America, and Chinese university students.^22,23 Since the age range of Chinese professional athletes is closely aligned with that of the student population, this structure might reflect the general characteristics of youth. Drawing from prior research, these two factors were labeled as sleep quality and sleep efficiency.²² This naming is derived from the content of the components. Interestingly, if the subjective nature of the responses of these components is taken into account, it can also be found that the former’s components leaned towards athletes’ subjective experiences and estimations of sleep, while the latter’s were associated with objectively calculable time.¹¹

The structural validity of the PSQI in a single-factor model is suboptimal. This finding aligns with the results of many studies. It is not an exception.^17,23 This phenomenon raises concerns about the practice of using total scores to evaluate athletes’ sleep quality. Given the clinical origins of the PSQI, some researchers have pointed out that its seven components lack a solid foundation, and integrating them into a single dimension is not a constructive approach.⁴⁴ Data also support this perspective. For instance, researchers using the Rasch model to analyze the PSQI found that a usable single-dimensional scale was achieved only after deleting and modifying many items.⁴⁵ Therefore, using a two-factor structure to calculate PSQI scores might be more suitable for non-clinical athletes.

The PSQI exhibited a characteristic of diminishing test-retest reliability with shorter intervals when utilizing a one-month timeframe. Prior research suggested that insomniac patients demonstrated better test-retest reliability with a two-day interval but not over longer durations.²⁶ This result sharply contrasted with the phenomenon identified in this study. However, adjusting the PSQI’s timeframe to one week enhanced the test-retest reliability of both total scores and individual dimension scores.

The observed phenomenon in this study may be attributed to the daily routines of non-clinical athletes. Individuals tend to respond to the PSQI based on their predominant life circumstances, such as workdays, or adjust their answers considering weighting factors.⁴⁶ This tendency is likely to be evident among athletes, given the presence of “social jetlag”, akin to office workers, involving factors like alarms and fixed training days. Furthermore, events such as training fatigue, adjustments, and participation in external competitions can significantly impact athletes’ daily routines over a period.^1,5,47,48 Therefore, changes in their short-term sleep quality are more likely and may be reflected in PSQI responses. Conducting a weekly retest within a one-month timeframe allowed for testing on similar training days. In contrast, a two-day retest might introduce influences from changes in weekly training schedules, potentially affecting reliability.

Concerning convergent validity, the PSQI demonstrated good validity on subjective reporting indicators in a one-week timeframe. However, objective measures of sleep quality did not exhibit the same consistency. The PSQI is considered the gold standard for subjective sleep quality assessments and serves as a criterion for numerous self-reported sleep measures (Fabbri et al, 2021), so good validity on subjective measures seems evident. However, a circular argument arises here - using scales that referenced the PSQI to assess its convergent validity or constructing scales during which the PSQI was used for reference. Nevertheless, this limitation was addressed by supplementing daily sleep diaries in the tracking process. Concerning objective indicators, the research found a correlation between PSQI scores and accelerometer results in clinical patients, a relationship absent in healthy populations.⁴⁴ Considering the clinical origins of the PSQI, it might be more aligned with clinical patients in certain indicators, leading to differences in convergent validity between clinical and non-clinical populations.

Athletes’ PSQI results within four weeks were unstable, indicating the PSQI’s capability to capture short-term variations. Results within a one-month timeframe were more closely related to the recent two-week single-week results. Within a half-month timeframe, they were more closely associated with the recent one-week results. This phenomenon may confirm interference related to “social jetlag”, impacting reliability. It suggests that athletes’ responses are likely influenced by the most recent 1–2 weeks. However, when incorporating scores for sleep medication, the relationship between total scores across four weeks remains relatively stable. Stable medication usage may dampen sensitivity to sleep changes in total scores, possibly linked to the original clinical nature of PSQI use.

Concerning convergent validity within a one-month timeframe, the convergent validity of total scores and sleep quality dimension were better on subjective reporting indicators. In contrast, the convergent validity of sleep efficiency dimension only occasionally reached acceptable levels in single-week measurements. This outcome suggests that the PSQI total scores may more closely reflect individual sleep quality rather than a comprehensive outcome. The inherent limitations of subjective reporting in the sleep efficiency dimension may be a primary factor. Studies have found that training fatigue can significantly impact the measurement results of physical activity records, precisely recorded by devices.⁴⁰ In contrast, the time reported in PSQI might mainly be based on daily schedules and subjective feelings, further diminishing responsiveness to changes and resulting in less stable reflections of sleep characteristics recorded objectively.

In general, the efficacy of employing the PSQI among athletes necessitates consideration of both the questionnaire structure and the measurement timeframe. This study identified a two-factor model comprising six components when applying the PSQI to athletes. Within these dimensions, the sleep quality component demonstrated robust convergent validity, likely exerting a dominant influence on the measurement outcomes, while the sleep efficiency dimension exhibited suboptimal convergent validity. Concerning the timeframe, the conventionally employed one-month interval is susceptible to the influence of recent states, whereas the one-week interval is less prone to such effects. Based on these findings, for non-clinical purposes involving the use of the PSQI with athletes, it is advisable to (1) compute scores separately for each dimension, emphasizing the sleep quality dimension; (2) utilize a one-week response timeframe; (3) supplement with Actigraphy or consensus sleep diaries when the research focus is on sleep efficiency;⁴⁴ (4) exercise caution in interpreting results, considering they may reflect recent states rather than the overall state throughout the entire timeframe.

While this study provided a comprehensive analysis of the issues surrounding the use of the PSQI among Chinese athletes, offering new evidence regarding its reliability and validity in this population and suggesting usage recommendations to enhance effectiveness, it is subject to several limitations. Firstly, the study was solely conducted with a sample of Chinese athletes, lacking cross-cultural comparisons. Previous research on the validation of sleep quality assessment tools not intended for clinical use has indicated potential cultural differences in the scale’s factor structure, particularly regarding the perception of sleep quality.³⁵ Future studies should further consider these cultural influences. Secondly, this study treated Chinese athletes as a homogeneous group. It did not specifically examine high-level athletes, such as Olympic athletes, or athletes with multiple identities, such as student-athletes, whose sleep quality has received more attention from researchers. Future research could differentiate between various types of athletes and conduct more targeted investigations to unveil more nuanced population differences. Lastly, in terms of methodology, to enhance the willingness and cooperation of athletes and coaches, this study did not adopt the approach of collecting objective sleep quality data continuously for seven days or longer, as seen in previous studies.^21,49 Although the duration of the Actigraphy recording in this study met technical requirements, its reliability and the information it provided are inevitably weaker than the continuous 7-day or longer recording method. Recording periods shorter than seven days lack the capacity to uncover potential phenomena such as “social jetlag” over long-term tracking, thus reducing the explanatory power of the results. Therefore, future research could consider supplementing with commercially available wearable devices proven to be effective for long-term monitoring to provide additional objective evidence for the effective use of the PSQI.^50,51

Conclusions

The Pittsburgh Sleep Quality Index (PSQI) revealed a two-factor structure among athletes, consisting of six components representing sleep quality and sleep efficiency. The sleep quality dimension demonstrated robust validity, potentially influencing the overall measurement outcomes of the PSQI, while the sleep efficiency dimension exhibited suboptimal validity. The test-retest reliability within the traditionally employed one-month timeframe was unstable and susceptible to recent short-term states’ influence. In contrast, a one-week timeframe showed better reliability and validity. When utilizing the PSQI with athletes, it is advised to focus on scores derived from the sleep quality dimension, adopt a one-week response timeframe, supplement with objective measurement tools, and exercise caution in interpreting the results.