E-learning - increasing member engagement while saving money
Success achieved
How the Association of Accounting Technicians is supporting their student members with e-learning, by adapting and adding new content, saving money and increasing member engagement
- Website
- www.aat.org.uk/find-a-course
- Project dates
August 2015 – July 2016
Project team
Learning Services:
- Amanda Healey-Browne, Course Leader: BA Accounting (with CIMA Advanced Diploma)
- Luke Cox, Learning Services Technologist
- Michael Beckley, E-learning technologist
Product Development:
- Rachel Kellett, Head of Qualifications and Product Development
- Theo Ege, Qualifications Development Manager
- Robyn Ford, Qualifications Development Manager
- Jodene Murphy, Project & Programme Management Office (PMO) Manager
Tools & Systems
- Camtasia
- Articulate Storyline
Introduction
AAT eLearning is included in membership fees. Once a student / training provider / member has their membership number they can log into the AAT site and access the range of support material.
AAT eLearning content is available through any web browser and it’s available in HTML5 format, which makes it easily readable on mobile devices.
Articulate provide a free app, which you are prompted to download. Once you download the app, all e-learning that you view is saved into an offline library.
We have just improved the functionality of the Greenlight test system so that the test is adaptive and resizes across different devices.
AAT eLearning content is compliant with Section 508 Accessibility guidelines which includes support for JAWS readers and full keyboard navigation.
Discovery
AAT launched online learning materials to support students studying the Accounting Qualification in 2011. Since then, the range of material has grown and online learning now supports all 15 AAT qualifications. The online learning material is designed to support students with areas of the curriculum that are considered to be hard, and are not designed to teach the qualification.
Since 2011, there have been two main online learning types: knowledge based e-learning produced via a rapid development tool, Articulate, and diagnostic test questions called Greenlight tests.
In September 2016, AAT planned to introduce new qualifications. These qualifications assess students differently by having a real-world focus and assessing the student synoptically. So, how could AAT provide supplementary support to cover these new qualifications?
In the 2015 e-learning survey, over 80% of students wanted to see how accounting topics are applied in the real life working environment. Students also wanted advice on how and what to study.
When students were asked what they thought of AAT e-learning, the response overall was good, with some interesting comments such as: ‘Useful for what I need but could be made more interesting and colourful. Can be a bit dull and cause me to skim read and lose interest.’
When asked what other content students would like, we had comments such as:
- ‘bite-size video lectures explaining the trickiest topics’
- ‘practical examples of how topics translate to the work environment’
- ‘A person led module, like the webinars but a little bit on each module’
- ‘Examples of accounting in real life situations.’
Current e-learning should also be reviewed to remove content that is considered to be instructionally poor, badly written, weak linkage to unit or of poor sustainability.
Objectives
The following targets were set for the production of AQ2016 supplementary learning material:
Budget
This project came in under estimated budget. We were able to make significant savings with content reduction, and moving some of the e-learning and all of the video production in house.
Reduction of poor historic content
We removed 29 existing e-learning modules. Modules removed were in tax subjects which are difficult modules to maintain due to continuous changes in the finance acts. Other modules removed were those no longer relevant to the new standards.
Timescales
The project was started in September 2015 and delivered in July 2016. Online content was then uploaded to the web service in time for tutors to review content in August 2016.
Development
Included 67 updated e-learning modules together with a further 20 real life scenario e-learning modules. In addition we delivered 54 standard Greenlight tests, 3 lucky dip Greenlight tests, 51 key calculations, 6 student and tutor introduction videos and 2 business communications modules. This totals 203 separate products.
Activities
To enable AAT to move forward and support our new qualification, we ran surveys to gain insight into what students liked about our current offering, what they didn’t like, and what else they would find useful. We visited our students at Training Providers to see how they use our offering and finally, we talked to the Training Providers themselves.
Student feedback 2015
A student survey was undertaken in March 2015. Over 250 students participated across three levels, 2, 3 and 4. Similar questions were asked as those asked in 2012 to compare and contrast. In 2015, nearly all students had accessed Greenlight and e-learning content; a significant increase from the 2012 figure of 61%.
In the March e-learning survey, over 80% of students wanted to see how accounting topics are applied in the real life working environment. Students also wanted advice on how and what to study.
When students were asked what they thought of AAT e-learning, the response overall was good, with some interesting comments such as: ‘useful for what I need but could be made more interesting and colourful. Can be a bit dull and cause me to skim read and lose interest.’
Much of the feedback that we received requested that knowledge should be conceptualised exemplified through the use of real-life examples.
When asked ‘What other content would you like AAT to produce?’ students responded:
- 'practice assessments’/‘greenlight questions’
- ‘bite-size video lectures explaining the trickiest topics’
- ‘video examples with opportunities for comments to be left with questions relating to the content’
- ‘practical examples of how topics translate to the work environment’
We reviewed offerings from similar organisations and looked at how our new qualifications had changed and the impact this would have on how training providers deliver the qualification. We also looked at developments in the field of online learning, particulary around:
Work examples
The Khan Academy has popularised working out complex mathematical problems by using recorded calculations with up to 6 million unique hits a month. ‘Working out the math’ is a common issue for accounting students and the idea of what feels like ‘one-to-one’ tutoring is an attractive one.
‘Teachers have long known that one-to-one tutoring is effective...’ ‘What Bloom found is that students given one-on-one attention reliably perform two standard deviations better than their peers who stay in a regular classroom.’ Thompson, C., 2011.
The method of recording is very sustainable using an application called Camtasia, with no overhead on production costs. Authors can record the worked example themselves into a styled template, while the production team, improve the audio and any mistakes the presenter has made. We decided to use the worked example idea for our new product ‘Key Calculations’.
Real world scenario based learning
‘Guided discovery” learning methods, such as scenario-based e-Learning, have been proven to be more effective than pure “discovery learning” because they provide guidance, structure and focused-goals.’
‘Learners are more motivated by scenario-based e-Learning than by traditional instruction.’ ‘Scenario-based e-Learning accelerates expertise.’
We decided to use the real world scenario approach on our new product, ‘Real life scenarios’, using the application Articulate Storyline.
We also looked at our processes and how much budget and resources we spent on development. We knew that we could cut production costs by using Camtasia and video shot and edited internally. We changed our instructional delivery methodology
to one that is more problem solving and real world scenario based.
We also looked at our processes and how much budget and resources we spent on development. We knew that we could cut production costs by using Camtasia and video shot and edited internally. We changed our instructional delivery methodology
to one that is more problem solving and real world scenario based.
It was proposed that students should access a range of learning materials that suit a range of learning styles. Those materials should form a learning pathway for the student, which begins with an introduction video.
This introduction video, discusses some of the key changes that a student will face studying the new standards. It is voiced by a student. Tutors also have their own video that helps them to consider topics such as synoptic assessment, employer engagement, grading and so on.
Proposed AQ2016 Content
Update current AQ2013 e-learning and Greenlight content.
- Optimize Greenlight environment in HTML5 adaptive framework for mobile
- Greenlight dashboards to incorporate student statistic performance for multiple attempts at any one test, as well as comparative statistics against the performance of other students.
- Modify question scoring in Greenlight to allow for partial scoring
- Create new ‘lucky dip’ test type which pulls questions from tests across a level to create a board level test.
- Current e-learning should be reviewed to remove content that is considered to be instructionally poor or badly written.
The student then selects the module that they are currently studying. Within a module, the student now has a range of content that they can learn from.
We have updated our existing e-learning content for AQ2016. These modules are knowledge based and help the student to understand the theory behind the topic.
We have updated our existing Greenlight tests. These Greenlight’s test a student’s knowledge in preparation for their assessment. We have brought in new gamification features this year such as Leaderboards, which show a student how they compare against others taking the same test. The tests are now mobile friendly so display on a mobile phone and even a smart watch!
We have also made improvements on the scoring. Previously, questions often had multiple answer fields, but only one mark attached to them. We have now attached marks based on number of answer fields.
We have also created a new type of formative test – a lucky dip test. Built in the Greenlight environment, the lucky dip test concatenates questions across the level into one big level test. This is not supposed to be a synoptic test, but does support the idea of being tested on different topics in one test.
Real life scenarios are new and use the idea of scenario based learning to support active problem or case based learning strategies. It involves the student working their way though a storyline and solving issues and obstacles in their way. The scenarios usually span several modules – because topics don’t happen in isolation to each other. This also allows us to support the idea of synoptic assessment, which looks at the gaps between topics and their interrelationship.
Key calculations – recorded Camtasias
Annotated videos of tutors working a calculation. Up to three per unit (depending on relevancy), of a maximum of 5 minutes long.
We have created new Key Calculations. Like the work pioneered by the Khan Academy, this is based on information theory, where we use language to break down a problem which is graphically illustrated into a small quantifiable chunk. It also builds on the idea of a virtual tutor – so we have used the same author throughout each level. These are also easy and cost effective to produce.
Language of Accountancy – communication skills – e-learning
Student support to cover business communication in the workplace including ethics and professionalism.
Levels 3 and 4 only. Finally, we know that an accountants skill in the future is not just to calculate numbers. AQ2016 clearly identifies that accountants needs to be able to communicate effectively, to write and to explain complex terms easily. The AQ2016 assessment contains more written elements then previously, so to support this change, we have created new modules for levels 3 and 4 on Business communications. These are based on learning theory that allows the student to identify and reflect on what makes good or bad communication, and then gives opportunities for practice. The modules focus on different areas, such as emails, reports, meetings and presentations.
Launch of AQ2016
All student content was made available to tutors on the 1 August 2016. It was made available to students on 1 September 2016.
Challenges
Producing video
AAT had never produced video internally before, we have always outsourced this function. Major challenges in this area were:
- Finding a quiet area to record; in the end we used a meeting room and did as much of the recording as we could at weekends and in the evenings.
- Ensuring that Brand were happy with the style and quality of video; this was a learning curve for both Brand and Learning Services, as we were unaware of each others requirements. In the end, some video had to be reshot or a large amount of post production time was spent geeting the video right.
- Embedding video in Articluate; this worked well for the shorter introduction videos, but for the bigger Business Communication pieces, we had to look at file sizes to enable effective functionality.
Sourcing imagery
AAT had previously used an image bank of hand drawn assets to illustrate its e-learning. For the Real life scenarios, we wanted to use images of real people. The Brand deparment wanted to ensure authenticity, and use AAT members. This caused significant overhead in organisation, as we had to recruit members via social media and email, then coordinate the photo shoot while arranging the equipment and location (AAT offices).
A standardised approach to scenario writing for authors
As the Real Life scenarios and Key Calculations were a new product, we had not set out the product specification fully in terms of length, introduction and outro scenes and resources. In the end we went for a variety of approaches which we aim to test and evaluate later in the year.
Achievements
In 2012, AAT won the Gold award for the ‘best use of rapid eLearning’ at the eLearning Awards. This has helped to raise AAT’s profile with students, training providers and members on the quality and range of learning methods available. In 2014, AAT won ‘best successful virtual offering’ with the Association Awards.
A survey of 253 AAT students was carried out in 2015 to gauge the impact of AATs learning resources. Those students were evenly spread across levels 2, 3 and 4.
In 2012, 61.7% of all students saying they had tried at least one eLearning module. However, in 2015, 94.7% said they had access AAT online learning.
The area on the AAT website that contains the online learning content has the 3rd highest hit rate averaging 256,411 unique page views a month. On average, an existing e-learning module generates 178 unique page views per month. Even though the new online content has only been in operation for students since the beginning of September, we have had between 70-105 unique page views on our new modules.
What would we do differently?
Writers workshops/ kick off meetings
We have already identified that a major challenge was having a standardised approach to scenario writing for our authors. One of the ways that we could have addressed this was to run writers workshops, similar to a kick off meeting, where authors could be given examples of the types of scenarios, the length, the narrative and dialogue. It would also have been an opportunity to address the issue of attached tasks and resources. It would have ensured that all authors understand how to use the templates effectively, and while authors knew screen types available, a workshop would have allowed exploration of the limitations and functionality.
Project update meetings
All products that Learning Services produce need to be signed off by the Business Owner (in this case, Product Development) and Brand. Learning Services made assumptions on the level of knowledge in the business towards production process, and as a result, changes which should have been asked for at the beginning of the project were asked for towards the end of the project. This caused some set backs on delivery.
Feedback & Testimonials
‘AAT’s new e-learning and support for AQ2016 is fantastic. We especially love the Key Calculations’.
Angela Renshaw, Damar training – AAT’s training provider of the year (small) winner
