Evaluation of the Fetal Alcohol Spectrum Disorder Hub Australia website

Introduction: To evaluate use and utility of the Fetal Alcohol Spectrum Disorder (FASD) Hub Australia website. Methods: Online REDCap user survey incorporating the Website Evaluation Questionnaire, open ended questions, Google metrics data and an accessibility audit. Results: Seventy-six participants: researchers (32%), health professionals (29%) and policymakers/advocates (16%) completed the survey. Most were from Australia (95%) and were likely or very likely to recommend the FASD Hub to colleagues (92%), friends (74%) and patients (72%). The mean Website Evaluation Questionnaire score was at least 3.45/5 for all dimensions (ease of use, hyperlinks, structure, relevance, comprehension, completeness, layout, search option); range 3.45 (search option) to 4.04 (relevance). Participants found the content trustworthy (92%) but wanted more information for, and to support, Aboriginal and Torres Strait Islander peoples, and improved search capacity. Google metrics identified 25,534 unique users over 6 months (82% new users); 83% aged 18 – 44 years, 72% female and 35% international. Conclusions: Users found the FASD Hub accessible, authoritative and useful and


| INTRODUCTION
Recent data from pregnancy cohorts suggest that over 60% of Australian women drink alcohol during pregnancy, risking adverse pregnancy and infant outcomes [1,2].Fetal alcohol spectrum disorder (FASD) is caused by prenatal alcohol exposure, is potentially preventable and is being increasingly recognised by clinicians in Australia, as indicated by reports of cases to the FASD Australian Registry [3].Based on the Australian Guide to the Diagnosis of Fetal Alcohol Spectrum Disorder, the diagnosis of FASD requires evidence of prenatal alcohol exposure and severe impairment in three or more domains of central nervous system structure or function [4].The diagnosis has two sub-categories: FASD with three sentinel facial features; and FASD with less than three sentinel facial features.FASD is sometimes accompanied by other congenital anomalies and growth failure.It is associated with severe learning, behavioural and/or mental health challenges with lifelong impacts [4].Given the reliance on the internet from professionals and the general public, the amount of health information available on the internet, and its potential to affect health care decisions, it is important that such information is easily accessible, relevant and evidence-based.
In the absence of a single source of information on FASD and alcohol in pregnancy for health professionals, researchers and policymakers in Australia, the FASD Technical Advisory Group identified the need for a dedicated website.This was included as a key priority in the Australian Government Department of Health's 2013-2016 strategy to address diagnosis, management and prevention of FASD.In response, the Department, under the Substance Abuse Prevention and Service Improvement Grants Fund, provided funds to The University of Sydney in 2016 to develop the FASD Hub Australia ('the FASD Hub') which was launched in 2017 (Figure 1).The FASD Hub aims to be a unique resource providing locally relevant Australian information about alcohol and pregnancy and FASD.It is intended primarily for health professionals, researchers, policymakers and educators, but is also relevant to individuals and families with FASD, Aboriginal and Torres Strait Islander people, and women who are pregnant, planning pregnancy or breastfeeding.Funding received for the development, maintenance and expansion of the FASD Hub and the timeline are outlined in Figure 1.
Consultation with potential stakeholders (including members of the National FASD Technical Network, health professionals, researchers, government representatives, NOFASD Australia and consumers) informed initial development of the site information architecture, wireframes, design specifications and graphic design, which in turn guided the technical build, testing and modifications to the final site.Over 200 individual webpages were written and almost 300 web resources were collated, reviewed and approved.
In 2018, after the FASD Hub had been operational for several months, an email survey was sent to 164 stakeholders on our distribution network (health professionals, Aboriginal health workers, justice professionals, educators, service providers, researchers, parents and caregivers, government, policymakers).The survey asked opinions about the ease of understanding information on the Hub, whether information was trustworthy, evidence-based and easy to find, whether users would recommend the website, whether the information met their needs, what the top three most useful sections were and what suggestions they had for improvement.A total of 60 responses were received (35% were health professionals, 17% researchers, 17% parents/carers, 7% from the health policy sector, the remainder unspecified).Of the 60 respondents, over 80% found the website easy to use and the information trustworthy, evidence-based and easy to understand, while over 70% said it was easy to find what they were looking for and the website met their needs.The top three pages accessed related to understanding FASD, managing FASD, and diagnosis and assessment of FASD.Suggestions for improvement included increasing the information provided on alcohol use in pregnancy, diagnosis in adults, training opportunities for health professionals, strategies for behavioural management in FASD and strength-based approaches to management.The need for plain language resources for culturally and linguistically diverse and Aboriginal and Torres Strait Islander peoples was identified.These suggestions were incorporated where possible.
In 2019, the National Health and Medical Research Council Centre of Research Excellence in FASD, led by Telethon Kids Institute and The University of Sydney, formed a consortium that included Curtin University, Griffith University, Menzies School of Health Research, Murdoch Children's Research Institute, the University of Queensland, the Australian and New Zealand FASD Clinical Network (Clinical Network), NOFASD Australia, and the Foundation for Alcohol Research and Education.The consortium was awarded funding for the Hub in April 2020 for a period of 4 years, which was extended to 2026 and included re-evaluation of the Hub.In this paper, we report the results of our 2021 evaluation of the use, accessibility, content and usefulness of the FASD Hub using a survey and review of the site's use using Google metrics.

| Participants and recruitment
Individuals aged 18 years or more, including but not limited to health, justice and education professionals, researchers, policymakers, government employees, FASD advocates and people with lived experience, were emailed an invitation letter through our distribution lists, professional organisations and snowball sampling to complete an anonymous online survey through a link to a secure web-based software platform, Research Electronic Data Capture (REDCap), hosted by The University of Sydney [5,6].Consent was assumed if invitees undertook the survey.The survey was piloted with six colleagues who are members of the Hub target audience and found to be comprehensible and easy to complete in less than 20 min.
As a result of the snowball technique, no denominator data were available and hence, unable to calculate completion rates.Participants could choose to enter a draw for one-of-six AU$50 e-gift cards.

| Instruments
The survey consisted of 48 items: the 26-item Website Evaluation Questionnaire (WEQ) [7,8] and 22 additional questions, collecting quantitative and qualitative data on participants' background and role; their opinions about the website's accessibility, content and relevance; and suggestions for improvement, as were asked in the 2018 survey.Free text fields provided opportunity for open-ended responses to the following questions: What is your primary reason for using the Hub?; What content or features do you like best about the Hub?; and Is there additional content or features you would like to see on the Hub?
The WEQ is a validated instrument designed to evaluate the quality of governmental and informational websites that aim primarily to provide knowledge to users without commercial or entertainment motives [7,8].The WEQ has 26 items that measure the quality of three key website attributes: navigation (the information seeking process), content (the outcome of this process) and layout (the 'look and feel' of the website).The WEQ includes three items for each of the following seven dimensions: ease of use, hyperlinks, structure, relevance, comprehension, completeness and layout.The items are rated on a five-point Likert scale ranging from 1 ('strongly disagree') to 5 ('strongly agree'), with scoring for negatively stated items reversed.The mean score for each dimension is calculated (score range 1.00-5.00)and higher scores indicate higher quality [7].

| Google metrics
Google Analytics metrics were captured from 1 July to 31 December 2021 to document the number of website users, their demographics (including gender, age, country of origin), whether they were using the website for the first time and the mode of access (laptop, desktop, mobile).Google metrics identified the number of user sessions and pages viewed each session, the duration of sessions and page views, and the bounce rate (defined as the proportion of users who enter and immediately leave the webpage).Google metrics identified how users located the site and the time, day and month of use.Google metrics identified the most visited pages on the website and the number of visits to these pages per month.

| Accessibility audit
Many people with FASD experience problems with vision, hearing and visual-motor integration which may impact their capacity to access information on a website.Thus, an accessibility audit was conducted by the Centre of Accessibility Australia, a not-for-profit organisation, and the website was assessed using Web Content Accessibility Guidelines 2.1.

| Data analysis
The Statistical Package for the Social Sciences (SPSS, Armonk NY, IBM Corp.) Version 25.0 was used to calculate frequencies, means and standard deviations for quantitative data.The free text responses to the three open-ended questions were categorised by two independent researchers (Heather Jones, Natalie Eastwell).

| Ethics approval
The research protocol was approved by the University of Sydney Human Research Ethics Committee (HREC 2021/727).

| Survey
Seventy-six participants completed the survey.Their key roles were as researchers (32%), health (29%) and other professionals (8%), people with lived experience (8%), and policymakers, advocates and government staff (8%).Most users were in Australia (95%), with 5% in Canada or the United States.Most learnt about the website from colleagues (63%), professional organisations (24%), social media (17%) and internet search engines (9%).One-third (33%) reported using the website at least once a week, and most others used the website once a month (26%) or every few months (25%).Participants reported that they thought information on the website was trustworthy and authoritative (92%).
Website content considered the most useful was the Australian Guide to the Diagnosis of FASD (65%), fact sheets (50%), e-learning modules (40%), videos (40%), publications (36%) and the Service Directory (34%).Most participants reported they were likely or very likely to recommend the website to colleagues (92%), friends (74%), patients or clients (72%) and family members (66%).Key suggestions for website improvement were a better search function, more extensive Service Directory, additional information for and about Aboriginal and Torres Strait Islander peoples, and targeted public health messaging for youth regarding alcohol harms in pregnancy.
Mean scores for dimensions comprising the Website Evaluation Questionnaire are presented in Table 1.All dimensions scored at least 3.45/5.00.Dimensions rated most highly were relevance and comprehension.Dimensions rated as relatively lower quality were layout and search option, aligning with the suggestions for improvement provided as comments in the survey.

| Qualitative data
Most participants (72; 95%) provided a response to the question 'What is your primary reason for using the Hub?'.The majority of respondents sought information, resources and training opportunities regarding FASD for themselves or other health professionals, or to help support families.Others used the FASD Hub to identify health professionals, services, specialists or referral pathways for FASD assessment or to find research relating to FASD or alcohol use in pregnancy.
When asked 'What content or features do you like best about the Hub?' 51 participants (67%) provided a response.Many responses were general-for example, about the ease of navigation, clarity and professional presentation.The Australian Guide to the Diagnosis of FASD, the e-learning modules, the Service Directory, research publications and the videos were also commonly mentioned.One participant shared 'I think it lives up to its name, the "hub."It linked me into a lot of resourcesgave me a better view of the information landscape'.
A total of 13 participants provided information to the question 'Is there additional content or features you would like to see on the Hub?' Responses included the need for additional information and resources about and for Aboriginal peoples, diversity in the imagery, a better search capacity and a more extensive Service Directory.

| Google metrics
Google metrics for 1 July to 31 December 2021 (Table 2) showed that the FASD Hub had over 25,000 unique users in 6 months (82% being new users) and over 4200 visits per month for an average duration of 2 min.Visitors were from Australia (64.5%), followed by North America, the United Kingdom and New Zealand.Most users (72%) were female and 83% were aged 25-34 years.Most users (73%) found the FASD Hub through an organic search, indicating that the website has excellent search engine optimisation, which in turn reflects user experience, and most accessed the FASD Hub on a desktop computer.
T A B L E 1 Scores for dimensions of the Website Evaluation Questionnaire [7,8].

Dimension
Mean score (SD) Mean range of score Ease of use 3.70 (0.79) Users were referred by several other websites, including from government departments and academic institutions (Table 2).Apart from the home page, the Service Directory was the most visited page (1924 visits per month), followed by the Australian Guide to Diagnosis of FASD (950 visits per month) and a page called 'What can teachers do?' (576 visits per month).
Figure S1, Supporting Information, shows the total number of sessions per week.The maximum number occurred in the week beginning 6 September, that is, FASD Awareness Week.The website was most often used mid-week, with peak use on Tuesday to Thursday in the middle of the day.User behaviour within the website (how the user moves between pages) differed depending on how the user entered the website.Of all users who arrived at the home page, 51% continued on to a second page, with What is FASD?, Understanding FASD and the Services pages being most popular.Users whose landing page was not the home page show different and more selective behaviour.For example, 91% of users who begin their journey on the Services page generally did not progress further on the website before exiting.Of all users who begin on the What can teachers do? page, 80% exited and 20% continued to a second page, half selected to visit Common Difficulties and Tips for classroom management strategies.

| ACCESSIBILITY AUDIT
The audit identified 11 simple issues (e.g., insufficient contrast between text and background colours, insufficient use of transcripts for videos, documents not having their language marked) which were resolvable, and 26 major issues that required attention by the web developer and additional budget.These included a lack of ability to add alternative text for imagery across the website, lack of labelling of buttons and links, and errors on certain parts of the website when navigating the pages using a keyboard.

| DISCUSSION
Our evaluation of the FASD Hub Australia website has shown it to be well used by the target audiences (health professionals, educators, researchers and policymakers), relevant, useful and regarded as a reliable source of evidence-based information on FASD and on alcohol use in pregnancy.Few websites are formally evaluated-methods of evaluating them include questionnaires, observed website browsing, interviews and focus groups, and web usage analysis [9].Cost constraints precluded us using observed web browsing, interviews or focus groups.We chose instead to use a snowball technique to reach a variety of users of the FASD Hub with an online survey and to examine web analytics.The survey allowed collection of quantitative and qualitative information on users' experience and opinions when using the website in a natural setting and included a validated and reliable researchbased tool, the WEQ [7,8].The WEQ was specifically developed for assessing government and other informational websites.The seven dimensions of the WEQ have been found to measure website quality in a stable way in different websites, and the attributes assessed cover those listed by Allison et al [9] as important influences on user experience (ease of use, content, functionality, appearance, web criteria such as media and searchability, interactivity, satisfaction and loyalty).However, as Elling et al. [7] note, the questions are not specific for evaluating particular pages on the website, and they suggest using open-ended questions (as we have done) to identify pages most used and users' goals in visiting the website.
Based on the results of the WEQ, our respondents scored the dimensions of relevance and comprehension most highly, however all dimensions scored 3.45/5 or above, indicating good navigation, content and layout of the FASD Hub.The lowest score of 3.45/5 for search options was reflected in participants' comments regarding a need for an improved search capacity and substantiated by the accessibility audit.Survey participants found the Australian Guide to the Diagnosis of FASD [4] and the information on services useful and these pages were identified as among the most accessed in the Google Analytics data.
The Google Analytics data also indicated that the website was most often accessed from a desktop computer, with peak use on Tuesday to Thursday in the middle of the day, suggesting that the website is used for professional purposes during office hours.Most users arrived at the website organically through a search engine (73%), frequently Google, and 9% came through referral from another website, the top ones being health, FASD or research related.The FASD Hub's bounce rate of 62.8% is comparable to the standard bounce rate of about 60% for health care websites [10].Bounce rate and number of pages viewed per session are measures of audience engagement.The bounce rate indicates the proportion of users who enter and immediately leave the website after viewing only one page, by clicking back, closing the window, following an external link or entering a new URL, and typically ranges between 30% and 67% depending on the industry [11].The number of pages visited per session indicates the number of pages the user views before leaving the website.Our finding of two pages per session for visits to the FASD Hub is in keeping with the two to three pages per session that is considered a good benchmark for performance across industries [12].
A few participants made valuable suggestions for additional features and content, which we are addressing.We are seeking to improve the searchability of the website.In response to initial feedback, we sought the advice of Aboriginal colleagues on the FASD Hub Advisory Group, who identified areas of the Hub requiring better accessibility for Aboriginal and Torres Strait Islander web users, as well as the need for Hub resources specifically for and about Aboriginal and Torres Strait Islander people as a priority.They advised forming an Aboriginal and Torres Strait Islander Advisory Panel for this purpose, and this has now been done.Participants also called for a more extensive Service Directory.We aim to keep the Service Directory current and complete, but unfortunately there is a limited number of specialist clinics and practitioners diagnosing FASD across the country, and all those we are aware of are listed on the FASD Hub.We have taken note of, and will address where possible, the suggestion for more diversity in images.
The Hub's remit is chiefly to house Australia-focussed information and resources relevant to health professionals, researchers, educators and policymakers, although we aim to provide some information that is relevant for parents, carers and the general public, including messages about alcohol abstinence in pregnancy and breastfeeding, NOFASD Australia, a family-focussed organisation, provides excellent information and support specifically for parents and carers via their website (nofasd.org.au), and links those with lived experience with researchers and clinicians.
Our survey respondents were mostly from Australia and covered a range of key disciplines.However, the survey evaluation is based on a small sample size of users, which limits generalisability of the results and did not assess how useful or appropriate the website might be for those who are new to FASD and the FASD Hub.Furthermore, Google Analytics data showed that most (82%) of over 25,000 users over a 6-month period were new users.In contrast, one-third of survey respondents accessed the website at least weekly and 51% used it at least every few months.This suggests the groups differ and may introduce some selection bias into the result.Nevertheless, the survey includes a validated and reliable questionnaire, assessing the main attributes of a good website [7][8][9].A formal thematic analysis of the qualitative data may also have added to the value of the open-ended questions.The results are similar to those of the 2018 survey, indicating that the FASD Hub is continuing to deliver on its aims, and is providing us with valuable information with which to improve and expand the FASD Hub.
In summary, the FASD Hub is a unique resource internationally, intended primarily for health and other professionals, researchers and policy makers.Few such websites have been evaluated.Although it has some limitations, our evaluation incorporates a validated website assessment tool, google analytics and open-ended questions.It shows the website is well used and valued, the content is relevant, trustworthy and useful, and that the structure, layout and hyperlinks are acceptable.Suggestions for improvement, including a more sophisticated search engine, and more content relevant to Aboriginal and Torres Strait Islander peoples will guide current redevelopment of the FASD Hub and inform development and evaluation of similar health websites.
Google metrics for the FASD Hub Australia website (1 July to 31 December 2021).
T A B L E 2 T A B L E 2 (Continued)UsageTop pages by view (views in 6 months)