Methodology

This study applied the Tuning methodology that has proven to be applicable at global level. Tuning projects have been implemented in some 130 countries involving all world regions. Tuning Educational Structures is an academics-driven process, developed originally in Europe to implement the Bologna Reforms, which offers a universal approach to Higher Education reforms both at the macro-level of entire Higher Educational Institutions and at the micro-level of individual disciplines/subject areas. The Tuning approach consists of a model to (re-) design, develop, implement and evaluate study programmes for each cycle (bachelor’s, master’s and doctoral). Tuning introduced the concept of student-centred and active learning in transnational context in Europe 2001, and at later date worldwide for all three levels of learning. It also introduced the notion to distinguish generic and subject specific competences to be developed in alignment in every degree programme.

The Tuning Methodology offers a number of key characteristics that makes it adaptable to any context in Higher Education where curriculum reform – in its broadest sense – is undertaken. The most important elements are twofold.

  • Firstly, the inclusion at every stage of a range of actors – which may include academics, employers, students, professional bodies, and officials – who are enabled to engage in constructive and guided debate on key issues. This inclusive openness is vital if a reform is to be understood, owned and disseminated.
  • Secondly, the methodology is systematic and proceeds from programme initiation to quality assurance, and can be absorbed into the curriculum reforms that an institution or nation is undertaking. Each stage has well established and trialed ‘tools’ to facilitate discussion, all of which can be adapted to particular contexts.

The first stage of the approach looks at programme design: what content and which outcomes are appropriate at each level. This involves consultation with employers, officials, professional bodies, academics and students – former and current. There is no central coordination in this respect either at regional or at national level, although it is expected that the Tuning conceptual frameworks are taken into account. Institutions are autonomous in designing and delivering degree programmes.

Tuning serves as a platform for developing reference points, descriptors or standards – as part of conceptual frameworks – at sectorial /domain level as well as subject area level. Among the subjects for which conceptual frameworks have been developed are: Chemistry, Physics, Mathematics, Earth Sciences, Medicine, Nursing, History, Art History, Linguistics, Literary Studies, Law, Social Work, Music, Performing Arts, Creative Arts, Architecture, Education Sciences including Teacher Training, Business Administration, Sociology, different types of Engineering and Area studies and Gender studies (the latter two  are samples of interdisciplinary studies). Tuning has also developed frameworks for the academic domains/ sectors Engineering, Social Sciences, Humanities and Creative and Performing Arts.

Frameworks are relevant for making programmes of studies comparable, compatible and transparent in a national as well as an international context. These are based on identified and agreed learning outcomes and competences, distinguishing between generic or transversal and subject-specific ones.

Relevant in the setting of the 2nd phase of the Tuning EU-China study proved to be that in the period 2016-2018 Tuning developed in the context of the project Measuring and Comparing Achievements of Learning Outcomes in Higher Education in Europe (CALOHEE) the most sophisticated subject area frameworks available by aligning and merging the models of the European Qualifications Framework for Lifelong Learning (EQF for LLL) and the Qualifications Framework for the European Higher Education Area (EQ for the EHEA). While the first focusses on the outcomes and relevance of the learning process, the second targets the (quality of the) learning process. This new model served as the foundation for defining subject area based qualifications frameworks in the Tuning EU-China Study. In comparison to phase I this has been an additional step.

The second stage of the approach examines Teaching, Learning and Assessment and considers how these are best designed and effected in order for students to achieve the intended learning outcomes. This is a stage where academics work together, sharing good practice and experience within and across subject boundaries. Students are also useful participants here. An important element in this respect is assessment (evaluation) of student achievement for progression, in relation to overarching Qualifications Frameworks (QFs) if available, which set out level descriptors for each degree level and (ideally) each level within a degree.

The third stage of the approach examines credits and student workload. There is no ‘right’ way of allocating credits or calculating workload, but in the interests of national and international student mobility there are now guiding parameters that need to be taken into consideration.

Finally, the last and fourth stage of the approach relates to Quality Assurance and Enhancement. It rounds-up the reform process. Tuning applies a six step-by-step approach applying the Tuning Dynamic Quality Assurance circle which distinguishes feed backward and  feed forward evaluation: what does work and what not and what to expect in the near future. Tuning contributes to the development and enhancement of high-quality competitive study programmes by focusing on fitness of purpose (to meet expectations) and fitness for purpose (to meet aims).

This methodology played a central role at the four meetings/conferences covering the period 2016-2019 in steering the discussions, collecting data and arriving to results. For the first phase a slightly different approach was applied , although it covered many of the same elements. What both phases had in common was the survey on the importance of generic and subject specific competences and the level of achievement in the learning process. This survey was also applied in phase I as a basis to define the meta profiles / conceptual subject area based frameworks for Business Administration, Civic Engineering and Comparative Education. Education was limited in practice to the specialized academic field of Comparative Education, which is taught at Master level. In the second phase, which covered the subject areas of Nursing, Information and Computing Sciences and Transport and Logistics, the outcomes were input for constructing the subject area based qualifications reference frameworks.

The consultation among stakeholders for the first phase was held in July 2013. For the second phase the outcomes of the surveys were presented in October 2018.

Use was made of the Tuning online consultation model. It distinguishes different stakeholders groups as well as the responses of different groups and institutions. Like in other Tuning consultations the following stakeholder groups were identified: academics, students, graduates and employers. The groups were consulted about the relevance of a number of identified generic competences / general academic skills and a number of subject specific competences for work in the profession involved. The consultation for the first phase was organised by the International Tuning Academy through internet. Identified stakeholders received an invitation e-mail to participate in the consultation by going to the Tuning consultation website. The link to this website was included in the invitation e-mail. Each stakeholder group and participating university obtained unique passwords. The consultation for the subject area involved in the second phase was organised by the Chinese universities leading the working groups.

For the first phase consultation, passwords were distributed to the Chinese group leaders who organised the distribution of the questionnaires to four stakeholder groups. In practice both printed questionnaires and the online tool where used. The data of the completed paper questionnaires were inserted in the online database for analytical purposes. The number of responses per subject area should meet the minimum requirement of 400 completed questionnaires, to offer reliable results. For each of the subject areas all five universities participated in the consultation. The numbers of completed questionnaires were the following:

Generic competences (same for all 3 subject areas)

Business Education Engineering Total
Academics 107 66 134 307
Employers 143 230 109 482
Students 152 138 184 474
Graduates 149 213 123 485
Total 551 647 550 1748

The list of agreed Generic Competences is included to this report in Annex 1.

Subject specific competences

Business Education Engineering Total
Academics 107 61 108 276
Employers 141 196 98 435
Students 150 122 166 438
Graduates 144 188 119 451
Total 542 567 491 1600

 The lists of agreed subject specific competences of the three subject areas are included in Annex 2.

The standards were met in all cases, with the exception of the number of academics in Education. This should not be perceived as a serious obstacle, because the outcomes are still reliable given the fact that only five institutions per sector were involved in a limited field. It has also to be stressed that the required number of 400 responses per subject area was amply realized.

The data were analysed by a team of experts of the University of Deusto. The data were presented at the seminars in the form of power point presentations, which required one and a half hour per subject area. The Chinese data were also compared in those power points with the data of the Tuning consultation held in Europe in 2008. The power point presentations and the raw data were supplied to the group leaders to enable further and deeper analysis of the data.

According to the Tuning methodology, the consultation was based on different parameters, that is, first:

    • the degree of importance: the relevance of the competence, in the opinion of the stakeholder, for work in their profession,
    • the level of achievement: the achievement of this competence as a result of having taken this university degree.

To evaluate these two variables, the interviewed had to use a scale: 1 = none; 2 = weak; 3 = moderate; 4 = strong.

and second:

  • ranking of generic competences: based on the categorisation of the five most important ones according to academics, graduates, students and employers. The competence that was ranked highest in the survey was allocated five points, four for the second and so on, with one point for the last in the selection. If the competence was not chosen in the survey, it scored zero points.

See for the templates of the Generic and Subject Specific on-line questionnaires Annex 3.

The outcomes of this consultation process were quite interesting. In the Chinese consultation most competences listed were thought to be important to very important. In general the achievement was thought to be one point lower on average. This gap is larger than in other consultation processes organised by Tuning. In a separate academic paper, already mentioned in the above as the 6th milestone, the data were presented in detail, further analysed and compared to the outcomes of Tuning surveys in other regions in the world, in particular to Europe. A first draft of the paper was ready in the first week of October, to be used for further debates at the third round of seminars in November/December.