Backup - Communicating with Recent Graduates: Making Sense of the GDS

The Graduate Destination Survey (GDS), with the Course Experience Questionnaire (CEQ) or Postgraduate Research Experience Questionnaire (PREQ) attached, is sent to all new Australian university graduates every year. The aim is to take a snapshot of what recent graduates are doing in terms of their post-graduation activities and to gather information from them about their higher education experience.

• Introduction
• GDS Overview
• Employment Rates and Starting Salaries
• The CEQ and the PREQ
• Conclusion, Appendix and References

Introduction

The Graduate Destination Survey (GDS), with the Course Experience Questionnaire (CEQ) or Postgraduate Research Experience Questionnaire (PREQ) attached, is sent to all new Australian university graduates every year. The aim is to take a snapshot of what recent graduates are doing in terms of their post-graduation activities and to gather information from them about their higher education experience.

The Graduate Careers Council of Australia co-ordinates the collection, processing and return of data to institutions prior to national reporting. The data files are used within institutions for course and careers advice, institutional planning and quality assessment.

Many people, beyond those who originally collected the data, are now making use of the information. The purpose of this paper is to act as an introduction to the GDS methodology, and the data gathered, to ensure that users understand the nature of the information and assist them in extracting figures useful and relevant to their needs.

Arguably, the challenge of a great deal of social research is to make sense of some (usually small) part of the world in a manner that enables us to understand and communicate about subjects that are often complex. This includes collecting, analysing and distributing the data and results in a manner that ensures users can make sense of the information and employ it as it was intended.

GDS Overview

Every year, the GDS form is administered to all new university graduates about four months after they complete requirements for a degree. The form for bachelor degree graduates or coursework postgraduates is coupled with the CEQ. A GDS form coupled with the PREQ is sent to higher degree research graduates.

The survey is managed within each institution by a Survey Manager who is responsible for the administration of the survey and (in most cases) the co-ordination of the distribution of the results and data.

Higher education sector representatives sit on the Survey Reference Group (SRG), which advises the GCCA on issues as they relate to the GDS.

A Standard Recommended Methodology (due for an update at the time of writing) is available as part of the GDS Manual (GCCA 2002). This details the timing of the survey, the tasks involved and the relationship between the institutional Survey Manager and the GCCA.

There is also a Code of Practice for the use of the data that addresses issues such as valid comparisons between fields of study and institutions and minimum cell sizes. This document is also available in the GDS Manual (GCCA 2002).

The GDS has been conducted nationally every year since 1972 and the CEQ was added in 1993 followed by the PREQ in 1999. The original aim of the GDS was to gather data to inform students about graduate labour market conditions and employment options and this remains a key focus of the survey.

The GDS form asks respondents about the course they have just completed, their labour market status, details of any employment they are in, and information about any further study being undertaken.

Data collected include:

·whether the respondent is in full- or part-time employment;

·whether the respondent is seeking employment;

if relevant

·the name of the employer;

·the location of the employer;

·the sector in which the employer operates;

·the area of industry in which the employer operates;

·the type of work the respondent is doing;

·the number of hours worked;

·the expected length of employment (permanent, short-term or ‘other’);

·annual salary; and

·whether the position is the respondent’s first full-time job.

Some features of this information (such as employment status and salary) are reported more frequently than others.

The most common themes for analysis centre on employment rates, starting salaries figures and CEQ results. Users should consider the way in which the GCCA conducts the core analysis of GDS, CEQ and PREQ data (as discussed in various GCCA reports). This will demonstrate the most useful and appropriate methods of analysis and include discussion of precautions about differing methods.

Employment Rates and Starting Salaries

By way of example, employment rates for the years 1990 to 2001 are shown in Table 1 (see Appendix). These figures are based on the group of respondents who are available for full-time employment at the time of the GDS. This group includes those who are in full-time employment and those who are seeking full-time employment. This latter group includes people who were not working and seeking full-time employment as well as those who were working on a part-time or casual basis and seeking full-time employment.

This mode of analysis ensures that graduates who were not in the full-time labour market (such as those in full-time study, those who were exclusively in the part-time labour market and those who were unavailable for paid work) are excluded from the figures. Labour market related analyses that do not exclude graduates who are outside the labour market should be avoided.

It can also be useful (depending on the analysis being undertaken) to take into account those graduates in full-time employment who held the same job during their study. Differing enrolment profiles at institutions can disguise dissimilar employment rates.

As with any analysis of data, the method chosen will require a consideration of what issue is being addressed and, of course, the method chosen should reflect acceptable social science practice. The annual GDS reports canvass a variety of method-related issues (GCCA 2002b).

Starting salaries

The figures the GCCA generally reports are based on Australian citizen and permanent resident bachelor degree graduates aged less than 25 and in their first full-time employment in Australia. This group is used to establish base-line starting salary figures to track changes in the earnings of new graduates.

Most comparisons are made against an annualised measure of average weekly earnings based on Australian Bureau of Statistics figures (ABS 2002) gathered at the time of the GDS. The annualised figure is based on male full-time earnings. Comparisons between salaries for female graduates with average weekly earnings figures for all females would see the former seemingly well paid, earning far in excess of the latter. This means that the comparison of starting salaries for female graduates with that measure may be seen as problematic. It is intended, however, that the average earnings figure be used as a constant, and not in a prescriptive manner.

However, Graduate Starting Salaries reports (see GCCA 2002a, for example) discuss earnings for the population aged 20 to 24 and refer to average weekly earnings figures for the combined workforce (males and females), and for the female workforce.

For the purposes of demonstration, Table 2 (see Appendix) shows graduate starting salaries from the 2001 Graduate Destination Survey. Asterisked cells indicate those where there are no, or fewer than ten, cases. The GCCA always warns users about small cell sizes in data analysis in order to avoid the misinterpretation of results.

Figure 1 (see Appendix) shows changes in starting salaries relative to average weekly earnings for the years 1977 to 2001. A zero point is not used as the analysis of graduates’ starting salaries concentrates on changes across time and not on an absolute comparison against the average earnings figure.

The CEQ and the PREQ

The Course Experience Questionnaire (CEQ)

The CEQ is a primary source of information for those charting quality measures in higher education. However, these figures should always be used with caution and insight. There is a time delay between the collection of such data and the production of results and this can dilute the usefulness of the information. For example, the graduate satisfaction figures produced below were gathered in 2001 from people who had completed their degrees in 2000. Respondents would have based their opinions on their experiences of the three, four or more years previous to that.

In that period, course managers could have been acting on prior CEQ feedback and fine-tuning their programs. So, for some courses at some institutions, the latest CEQ figures might not be a timely reflection of student experience, lacking the detailed items required to allow deeper analysis of issues specific to the institution.

However, it has always been argued by the GCCA that the instrument represents a macro-level overview of teaching quality and should form part of a suite of instruments used within an institution. The other components would be questions and scales developed within the institution that would get at more specific issues and put flesh on the CEQ figures. This gives institutions an incentive to develop internal quality assurance tools that lock the CEQ into their own systems and build on the data in a way that is meaningful for the improvement of teaching for that institution.

In addition to the issue of the timeliness of the figures, their relative complexity can make them difficult to interpret with accuracy. For example, Table 3 shows institutions ranked in terms the percentage of respondents who agreed or strongly agreed with Good Teaching Scale (GTS) items. The figures in this table seem to indicate that BondUniversity and CentralQueenslandUniversity received relatively high ratings in terms of the GTS. However it can be seen that these figures were based on very few responses (as were the figures for the next three institutions). Similarly, the lowest figure for agreement, for UNSW, was also based on few responses only. Figures based on such small response numbers cannot be assumed to be representative of the opinions of the whole group of graduates and should be ignored or treated with caution.

Even in cases where there are a sufficient number of responses, the figures need to be interpreted with caution. For example, while CurtinUniversity had an agreement figure close to the national average, its disagreement figure was very low. Curtin had a lot of cases fall into the middle ‘undecided’ category. Similarly, the agreement figure for the University of Adelaide was low. However, the disagreement figure was very close to the national average. Again, a lot of cases fell into the ‘undecided’ category.

At the other end of the table, Swinburne and VictoriaUniversity rated relatively well in terms of agreement but were near the national average in terms of disagreement. In this case, low percentages are found in the ‘undecided’ category. These examples all indicate that CEQ figures need to be treated with caution and a basic understanding of the intentions of the instrument and the methods of analysis. Users should consult the annual CEQ reports (GCCA 2002c) for background information.

Attempts to condense such figures to single digits or star ratings certainly have the effect of reducing the complexity of interpretation, but this is at a cost to the richness of the information.

The Postgraduate Research Experience Questionnaire (PREQ)

The PREQ is still in development in terms of optimal data analysis methods, however many of the issues discussed in relation to the CEQ are pertinent to the PREQ.

Conclusion, Appendix and References

In order to make full use of the data gathered via the GCCA surveys, analysts should:

·discuss their data needs with their institution’s Survey Manager;

·familiarise themselves with the GDS Manual and the Code of Practice for data use;

·familiarise themselves with the instruments;

·examine the relevant GCCA reports for detailed background information concerning data use and analysis; and

·contact the GCCA for advice where necessary.

The data gathered via the GDS, CEQ and PREQ represent an important resource for institutions as they work to improve the quality of their teaching. Appropriate application of the data will aid institutions in this aim, however an understanding of the survey methodology and most useful modes of analysis are needed to ensure that the full value of the data is extracted.

Appendix

Please download the relevant tables and graph from the links below:

• Communicating with Recent Graduates - Table 1
• Communicating with Recent Graduates - Table 2
• Communicating with Recent Graduates - Figure 1
• Communicating with Recent Graduates - Table 3

References

• ABS (2002) Average Weekly Earnings, Australia, November 2002, Australian Bureau of Statistics, Catalogue Number 6302.0, Canberra.
• GCCA (2002) 2002 GDS Manual of Operating Procedure, Coding and Data Entry Instructions, Graduate Careers Council of Australia, Melbourne.
• GCCA (2002a) Graduate Starting Salaries, 2001, Graduate Careers Council of Australia, Melbourne.
• GCCA (2002b) Graduate Destination Survey, 2001, Graduate Careers Council of Australia, Melbourne.
• GCCA (2002c) Course Experience Questionnaire, 2001, Graduate Careers Council of Australia, Melbourne.

http://www.gradlink.edu.au/content/view/print/867