Identifying seminal papers in the Australasian Journal on Ageing 1982–2011: A Delphi consensus approach

Authors


Abstract

Aims

The aim of this study was to identify seminal Australasian Journal on Ageing papers published over 30 years through a Delphi consensus process.

Method

The main data collection was a three-round Delphi consensus study with 38 past and current members of the Australasian Journal on Ageing Editorial Board, Editorial Team and Management Committee.

Results

Three papers were agreed as top-ranking. One of the top-ranking articles was also highly cited. One article was published in the 1990s, two in 2001.

Conclusions

While it is difficult to judge how well the top-ranking papers represent seminal papers arising over 30 years, these papers do represent three different research strengths in Australasia, they do span three different disciplines, and they do reflect some of the diversity that characterises ageing research in Australasia over 30 years.

Introduction

The first issue of Australian Journal on Ageing, the antecedent publication to Australasian Journal on Ageing, was published in February 1982. This first issue included articles by ongoing and current contributors, such as ‘Life-time earnings in Australia’ by S. Richardson and K. Hancock [1]; ‘Ageing, the family and public policy’ by D. Gibson and H. Kendig [2]; and ‘Perspectives on the transition from work to retirement’ by J. McCallum [3]: notably on topics of still increasing interest today. As noted by one of these contributors (Kendig) in 2010:

‘The Australasian Journal on Ageing is widely recognised as the leading peer-reviewed journal in ageing in our region and it has a widespread and influential readership in multi-disciplinary ageing research and its translation to policymakers, practitioners, and service providers …’

So, 30 years on from this first publication, how can we best highlight what the contribution of the journal has been to both the region and the field? One measure of impact and influence is the seminal works that have been published in the journal; that is, articles that were highly original, influential and important in the field, those papers that changed our understanding, and moved the field forward. The aim of this study was to identify the seminal Australasian Journal on Ageing papers published across 30 years through a Delphi consensus process [4].

Methods

Design

The main data collection was a three-round Delphi consensus study to identify the seminal Australasian Journal on Ageing papers from 1982 to 2011 (30 years of publishing). The Delphi survey technique is an iterative electronic survey method used to obtain agreement or consensus among a group of experts in a specific field on a particular issue through systematic multiple sequential rounds of survey administrations [4]. Additional data included a list of the highly cited papers over that time period, and rankings from the current editorial team (excluding LP) at Round 3 of the Delphi process.

Delphi participants

All discoverable past and current members of the Australasian Journal on Ageing editorial board, editorial team (i.e. editors and associate editors) and management committee from the last 10 years (excluding the authors) were invited to participate in Rounds 1–3. This group was deemed expert due to their high-level association with the Australasian Journal on Ageing. Details of names and affiliations were available from past editions of the journal and were checked against other Internet material and through known contacts to ensure the correct participant contact details. In Round 3, current associate editors were also invited to contribute to the Delphi process, using a separate form for data collection, so that biases and differences could be explored.

Identifying and classifying past published papers

All research papers from the Australasian Journal on Ageing published between 1982 and 2011 were included in the article sampling frame. All identified articles were organised into relevant themes from the abstract (or full text where needed) using a modified thematic analysis approach. To reduce response bias, all articles were de-identified in the survey: article title and abstract only were provided.

Highly cited articles (1982–2011) were identified firstly by searching Thomson Web of Science (from 1996 to 2011), then supplemented with a search on Google Scholar for older publications. Articles that had been cited more than 15 times were included in the highly cited list.

Criteria around ‘seminal’ for rating papers

A set of criteria was developed for rating papers as ‘seminal’ on the basis of their originality, lasting contribution to the field and quality of the research reported. Participants were asked to rate each abstract using a rating scale from 1 to 10 (where 1 was the lowest score and 10 was the highest score) on the criteria in Box 1.

Box 1. Criteria for rating articles as ‘seminal’

Originality and quality:

  • How original was the concept/question of this paper within Australasia (comprising Australia, New Zealand, New Guinea and the Pacific Islands) when published?
  • How original was the concept/question of this paper internationally when published?
  • How original was the methodology of this paper when published?
  • How much did this paper establish a new area of enquiry in ageing research or practice?

Influence:

  • How much did/will this paper influence research in Australasia?
  • How much did/will this paper influence research internationally?
  • How much did/will this paper influence policy in Australasia?
  • How much did/will this paper influence policy internationally?
  • How much did/will this paper influence practice in Australasia?
  • How much did/will this paper influence practice internationally?

Delphi procedure

Ethical approval was obtained through the Central Queensland University Human Research Ethics Committee. The Delphi process comprised three rounds, undertaken using an online survey developed in Survey Monkey®. The survey was hosted in Survey Monkey® so neither the research team nor other participants were able to identify individual participant responses.

All identified potential participants were invited by email to participate in the Delphi study, and provided with a link to the survey. Two email reminders were circulated per round. All participants were invited to participate in all rounds, regardless of prior participation in any round.

In Round 1 of the Delphi study, participants could choose to contribute to one or every theme. At the start of the survey, all themes were listed in alphabetical order, and themes were presented one-to-a page in the same order. At each new page, participants were asked if they would like to contribute to the theme featured on that page. If they chose to contribute to a theme, they were directed to the list of abstracts for that theme, and asked to rate each abstract using the criteria for ‘seminal’ as described above. Participants were asked to nominate if they were an abstract author and not to rate the abstract in that case.

For Round 2, LP and KR reviewed Survey Monkey® summaries to identify the top-rated papers across themes. Ratings for each article were totalled across participants and then divided by number of people rating to give a score from a possible 100. Only those articles that achieved a rating score of 70 or above were retained for Round 2. The survey was then modified for Round 2 to include only the top-rated abstracts, regardless of theme. At Round 2, the 15 most highly cited Australasian Journal on Ageing articles (with more than 15 citations) were also included to provide a richer sample of articles. In Round 2, participants were not required to rate the included articles, but to rank their top 10 articles across all themes, considering the standard definition of ‘seminal’.

For Round 3, LP and KR reviewed all ranked papers and assigned scores (that considered rank and number of raters) to identify the top-ranked papers using Excel spreadsheets generated from Survey Monkey® data. Round 3 participants (who included current associate editors) were asked to nominate their top five choices, in rank order, from the shortlist of highly ranked articles from Round 2. The aim of this round was to identify up to five agreed seminal papers.

Results

Delphi participants

Forty-four past and current members of the Australasian Journal on Ageing editorial board, editorial team and management committee from the last 10 years were identified. Email details were obtainable for 38 of these potential participants. Fourteen people participated in Round 1 (only three provided ratings, 8% response rate), nine people participated in Round 2 (24% response rate) and 14 people participated in Round 3 (33% response rate), including five of the current associate editors.

Identifying and classifying past published papers

From 1982 to 2011, 915 apparent research articles were discovered within Australasian Journal on Ageing. Overall, 99 were excluded as they were not research articles: 41 were conference reports or summaries; 19 were viewpoint or observation or invited commentary short pieces; nine were ageing organisation position statements; eight were reports on policy or meetings; five were conference abstracts; four were case studies or case reports; four were letters to the editor; three were book reviews; three were prose pieces; two were political policy statements; and one was a directory. Thus, 816 articles were included in the thematic analysis.

LP undertook primary thematic analysis of all abstracts, with confirmation by KR. Overall, 44 themes were identified, as listed in Table 1, with a range between 2 and 39 articles per theme (average of 18). A further six articles were excluded, as they were not research papers, leaving a final sample size of 810.

Table 1. Themes for articles published in Australasian Journal on Ageing from 1982 to 2011
ThemeNumber of articles
Aboriginal health and ageing4
Aged care services – resident health and advocacy31
Aged care services – assessment and models of care14
Aged care services – workforce5
Ageing in CALD communities39
Ageing research trends11
Ageism and attitudes to older people34
Carers – characteristics and impacts20
Clinical – assessment and professional roles15
Clinical – medical diseases23
Clinical – pain, sensory deficit, disability or oral health26
Community care – the client6
Community care – models and outcomes33
Cross-cultural comparisons in ageing9
Death and end of life12
Dementia/cognitive decline – aetiology and risk factors and epidemiology27
Dementia/cognitive decline – prevention and treatment5
Dementia/cognitive decline – care and carers32
Demographics of ageing10
Education in ageing20
Elder abuse20
Falls – epidemiology, risk factors, outcomes and prevention28
Finances – assets and retirement12
Finances –gender perspectives7
Finances – policy perspectives24
Grandparents and intergenerational perspectives5
Health-care services – ambulatory and acute22
Healthy ageing – physical32
Healthy ageing – psychosocial perspectives38
Housing choices and impacts27
Incontinence11
Lifelong disability – implications for ageing2
Lifelong learning and programs for older people11
Medicines and older people – treatment outcomes, adverse effects and quality of use31
Mental health and ageing27
Nutrition and ageing13
Physical activity, exercise science and older people12
Policy – national perspectives27
Policy – international perspectives5
Research methods – special considerations in an ageing population14
Research methods – measures development and testing33
Sexuality and ageing6
Technologies in ageing17
Transport and driving10
Total number of articles included810

Round 1 and highly cited articles

Overall, 14 themes were rated, and 18 articles scored 70 or over [5-22]. Fourteen articles with over 15 citations (range 16–34 citations) were identified [12, 21, 23-34], two of which overlapped with the Round 1 selected articles [12, 21], providing 30 abstracts for Round 2.

Round 2

Eleven articles were the most highly ranked from the 30 available abstracts (scores ranged from 26 to 73 from a possible maximum of 90); six of these came from the top-cited list [24, 27, 29-31, 33], and five from the Delphi Round 1 list [5, 6, 9, 18, 20]. Seven articles were published after 2000 [5, 6, 9, 20, 29-31], two were published in the 1990s [27, 33] and two in the 1980s [18, 24].

Round 3

Three articles were consistently highly ranked in the top three by the two participant groups (Delphi and editor), as shown in Table 2 [6, 9, 27]. There was no consistency of agreement between the two groups for articles ranked 4 or 5. Only one of the three top-ranking articles came from the top-cited list [27], having been cited 20 times according to Google Scholar; the other articles have been cited five [9] and three times [6], according to Thomson Web of Science. One article was published in the 1990s [27] and the other two were published in 2001 [6, 9].

Table 2. Ranking scores for Round 3 abstracts (published in Australasian Journal on Ageing from 1982 to 2011), by group and combined
Abstract [citation]Score Delphi (n = 9)Score editors (n = 5)Score combined (n = 14)
 1 [18]2.20.01.4
 2 [24]1.71.81.7
 3 [33]0.70.00.4
 4 [27]1.92.42.1
 5 [9]2.82.22.6
 6 [6]2.02.82.3
 7 [31]1.40.81.2
 8 [30]0.21.20.6
 9 [29]0.72.81.4
10 [20]0.70.40.6
11 [5]1.30.41.0

The references and abstracts for the three papers rated as ‘seminal’ through this process follow:

P.A. Atkin, T.P. Finnegàn, S.J. Ogle, D.M. Talmont. Prevalence of Drug Related Admissions to a Hospital Geriatric Service. Australian Journal on Ageing 1994; 13 (1): 17–21 [27].

217 consecutive admissions to an acute geriatric care facility in a teaching hospital were prospectively evaluated to assess the prevalence of drug related problems (DRPs) contributing to admission. DRPs were considered to include adverse drug reactions (ADRs), toxic overdose and poor compliance leading to therapeutic failure. The prevalence of DRPs on admission was 22.1% of total patients or 24.0% of those on prescribed medication at the time of admission. DRPs comprised 41 cases (85.4%) of ADRs, 1 case (2.1%) of exacerbated disease due to withdrawal of medication by the prescriber, and 6 cases (12.5%) of patient non-compliance. DRPs were significantly associated with both the total number of immediate pre-admission medications and the presence of a previous history of ADR. No other significant associations were found. The results suggest that prescriber behaviour, as opposed to patient behaviour, is a major contributor to DRPs and accordingly indicate the need for education of doctors about the risks of polypharmacy. DOI: 10.1111/j.1741-6612.1994.tb00627.x

Eyal Gringart, Edward Helmes. Age Discrimination in Hiring Practices Against Older Adults in Western Australia: The Case of Accounting Assistants. Australasian Journal on Ageing 2001; 20(1): 23–28 [9].

Objectives: To investigate the willingness of employers to hire older adults. Method: Measures of actual age discrimination were obtained through the use of correspondence testing. Unsolicited fictitious resumes were simultaneously mailed to 452 companies across industries in Western Australia. Resumes were similar in all job relevant information except the applicant's age that was either 32 or 57 and gender. Measures of discrimination were obtained through comparisons of the time for receipt, and the number of positive and negative responses across age and gender.

Results: Responses for 202 resumes were received from 154 companies. A significant three-way interaction between age, gender, and mailing condition was found for time for receipt of negative responses. The interaction showed that both younger males and older females were significantly faster to receive negative responses compared with either older males or younger females. Overall, younger applicants received a greater number of positive and a smaller number of negative responses compared with the older applicants.

Conclusions: The current study found evidence of actual age discrimination in hiring practices against older adults in Western Australia, with older women the most discriminated against. These trends of discrimination were different and of lower magnitude to those found in the US. Correspondence testing with unsolicited applications was shown to be a valid and sensitive tool for measuring actual discrimination in Western Australia. DOI: 10.1111/j.1741-6612.2001.tb00345.x

Louise M. Waite, Helen Creasey, David A. Grayson, Dorothy Edelbrock, John S. Cullen, William S. Brooks, Barney J. Casey, Hayley P. Bennett, G. Anthony Broe. Clinical Diagnosis and Disability Among Community Dwellers Aged 75 and Over: The Sydney Older Persons Study. Australasian Journal on Ageing 2001; 20 (2): 67–72 [6].

Objectives: Disability is an important predictor of health and community service utilisation. Understanding its pathogenesis has implications for planning of future health services. The aim of our study was to examine the contribution of systemic, psychiatric and neurodegenerative diseases to disability in an ‘old-old’ population.

Methods: 647 men and women over the age of 75 participated in the Sydney Older Person's Study. Disability in activities of daily living (ADL), instrumental activities of daily living (IADL) and mobility was assessed using self report, informant and clinician assessments. Diagnoses of systemic, psychiatric, neurodegenerative diseases were made by clinicians. Individual disease contribution to disability was assessed using multiple regression analysis. Contribution to disability by the groups of systemic, neurodegenerative and psychiatric diagnoses was assessed, using hierarchical regression. Results: Neurodegenerative diseases were the major contributors to ADL impairment. Systemic and psychiatric diseases played a role in IADL and mobility impairment, as did the neurodegenerative diseases. Of the neurodegenerative diseases, dementia/cognitive impairment and Parkinsonism/gait slowing particularly contributed to disability. Self-report under-identified the role of the neurodegenerative diseases in disability. It also introduced a gender effect, that the clinical measures did not share to the same extent.

Conclusions: Neurodegenerative diseases are important contributors to disability and assessments and diagnosis of neurodegenerative diseases should be included in disability assessments. Self report under-identifies the importance of this condition. DOI: 10.1111/j.1741-6612.2001.tb00356.x

Discussion

This study identified three agreed top-rated (not necessarily ‘seminal’) papers published in the Australasian Journal on Ageing across 30 years (1982–2011) through a Delphi consensus process [6, 9, 27].

The three top articles identified cross both thematic and discipline boundaries. One article reports the results of a clinical geriatric medicine focused study from the 1990s [27], examining medication-related admissions to hospital. This paper has been cited 20 times according to Google Scholar, and was drawn into the article sampling frame as a highly cited article. The two other articles were both published in 2001, and were nominated through the Delphi process. One comes from the discipline of psychology, and looks at age discrimination in employment [9]. The other article comes from an epidemiological perspective, and examines development of disability in an ageing cohort [6]. Given the breadth of research into ageing published by the Australasian Journal on Ageing, it would be difficult to capture this diversity in three or even five articles, but these three articles do provide good examples of that diversity.

A major limitation of the Delphi study was the lack of participation at Round 1 (less than 10% response rate), despite several reminders, which was disappointing from a group that was expected to have high investment in the journal. A major reason for this problem may have been the complexity of the task. Given the 30-year time frame, over 800 articles were identified, coded and grouped into 44 themes. However, each participant was asked only to rate at least two themes (less than 40 articles), which should have been manageable. Those who participated rated many more than two themes each. An alternative approach may have been to ask participants to nominate their own ‘seminal’ papers. However, this approach would have lacked standardisation and would have been open to as many biases as the poor response rate introduced. The approach used was expected to be more systematic and scientific than the open nomination method in intent, if not in outcome.

One strategy to counter this poor response rate was to increase the richness of the article sampling frame by including highly cited articles at Round 2. Response rates at Rounds 2 and 3 were much higher than at Round 1 (24 and 33%, respectively, which are relatively high for online Delphi processes) [35, 36]. Interestingly, the Delphi nominated list and the highly cited list had two articles in common, providing some confidence in the final list of articles for Round 2. At Round 2, about 50% of articles ranked highly came from each source, and at Round 3, two of the three ‘seminal’ articles were from Delphi Round 1. The final result would have been very different if the process had started with only the highly cited papers, another alternative approach.

It is difficult to judge how well the selected three papers represent what might be a wider view of the seminal papers arising from 30 years of the journal, or whether they best highlight the contribution of the journal to both the region and the field; and a different group of participants may have decided on quite a different final three. However, these papers are substantial contributions to research in ageing in Australasia; they represent three different research strengths in Australasia, they span three different disciplines, and they reflect some of the diversity that characterises ageing research in Australasia over the past 30 years.

Ancillary