Warehouse Floor Paint in Solas #Warehouse #Surface #Painting #Solas https://t.co/o6uEC79Nzy
— Marking Contractors (@markingcontract) February 28, 2019
Car Park Markings in Buttershaw #Carpark #Line #Marking #Buttershaw https://t.co/KhK8qLwy4L
— Marking Contractors (@markingcontract) February 28, 2019
Road Markings Meanings in Bedfordshire #Road #Line #Meanings #Bedfordshire https://t.co/P59m4FPhZq
— Marking Contractors (@markingcontract) February 28, 2019
Car Park Painting in West Lothian #Parking #Bay #Colour #Coating #West #Lothian https://t.co/Gz7Q5Mo4O0
— Marking Contractors (@markingcontract) February 28, 2019
Car Park Lining Services in Harbridge Green #Car #Park #Line #Marking #Harbridge #Green https://t.co/qrrWGr4gPp
— Marking Contractors (@markingcontract) February 27, 2019
Road Line Marking in Derbyshire #Roadway #Line #Markings #Derbyshire https://t.co/p9XSvvzyyA
— Marking Contractors (@markingcontract) February 27, 2019
Motorway Road Markings in Burnhope #Motorway #Marking #Burnhope https://t.co/1C2JWWw9rd
— Marking Contractors (@markingcontract) February 27, 2019
Car Park Markings in Nottinghamshire #Carpark #Line #Marking #Nottinghamshire https://t.co/eKdAXsWb9o
— Marking Contractors (@markingcontract) February 27, 2019
Thermoplastic Road Marking in Warrenby #Thermo #Plastic #Roadway #Markings #Warrenby https://t.co/W3O9YP6kOV
— Marking Contractors (@markingcontract) February 27, 2019
Thermoplastic Road Marking in Kent #Thermo #Plastic #Roadway #Markings #Kent https://t.co/1B2oDq909n
— Marking Contractors (@markingcontract) February 26, 2019
Thermoplastic Playground Markings in Armagh #School #Playground #Marking #Armagh https://t.co/opDNiycrTt
— Marking Contractors (@markingcontract) February 26, 2019
School Playground Painting in Falkirk #School #Play #Area #Paint #Falkirk https://t.co/CIx6qF4PJX
— Marking Contractors (@markingcontract) February 26, 2019
Without proper information and an understanding of wider concerns, governing bodies will continue to make poor decisions, argues Professor Norman Gowar, former principal of Royal Holloway, University of London
News that some universities are in financial difficulties should come as no surprise. It was entirely predictable. Most followed the same policy of chasing students to rake in profit from the fee increase. They thought the best way to do this was to improve the ‘student experience’. The nonsense of NSS and TEF encouraged this but most chose also to invest in marketing and massive building programmes, some to accommodate the growth they thought they should pursue, some as part of a marketing exercise. There was less investment in what really matters – the teaching and administrative staff at departmental level who actually engage with students.
Publicity about the level of vice chancellors’ pay has now drawn attention to the numbers of highly paid senior central administrators as priorities shift away from the coal face. No doubt we will soon be hearing horror stories about complex and dubious financial instruments taken on to fuel the building boom.
Not so widely discussed are issues of governance. Governing bodies cannot and should not be concerned with day-to-day running but they must share responsibility when things go wrong. But to exercise their responsibilities they need to be reassured that the information they receive and the choices presented to them have the widest possible support of the community and that the pros and cons revealed in such consultation are presented together with preferred recommendations.
With a shift from collegiate leadership to centralisation and top-down management the advice given to governing bodies is likely to be over-influenced by the views of a small cadre talking amongst themselves with their own group dynamic. The tragedy is that having taken the wrong strategic direction, the axe looks likely to fall on the very staff who have been neutered and unable to influence policy. Ironically they are the people who can restore fortunes by excellence in teaching and research. For many universities the fee increase could have been used to dramatically improve student staff ratios. This would directly enhance the ‘student experience’ and go to the heart of what a university is about. It would also please governing bodies by seeing a move up the league tables. But I wonder how many were given the chance to consider alternatives. This would have still left room for some much-needed improvements in the estate. It is extraordinary that with huge increases in income, debt to income ratios have risen. Keeping them constant would have been prudent whilst still giving plenty of flexibility.
Governance issues have also arisen over the issue of vice-chancellors’ remuneration. The problem is not only the composition of remuneration committees but the new managerial approach encouraging comparison with chief executives in the commercial world. Far better to reflect the actual job by grounding vice-chancellors’ salaries in the academic enterprise they lead and base them on fellow academics’ pay – say as a multiple of the average professorial salary. There is little evidence that the skills or levels of pay appropriate in the commercial world are relevant to a university’s leadership. One cannot help noticing that the nation’s most senior police officer is paid considerably less than most vice-chancellors and I guess that running the Met is a more challenging job.
When things go wrong governing bodies must be held to account but to do their job they need to be properly informed and understand the views and needs of the entire community in order to decide how best to guide the university in the head winds of changing government policy. Only then can they apply their wisdom from wide experience and in particular their specialist expertise to ensure probity and sound financial policy.
Car Park Lining Services in Heronsford #Car #Park #Line #Marking #Heronsford https://t.co/msw2NMHySL
— Marking Contractors (@markingcontract) February 26, 2019
Road Line Marking in Lincolnshire #Roadway #Line #Markings #Lincolnshire https://t.co/OPy2xCALAf
— Marking Contractors (@markingcontract) February 25, 2019
Warehouse Floor Paint in Penydre #Warehouse #Surface #Painting #Penydre https://t.co/6E3nou3tEi
— Marking Contractors (@markingcontract) February 25, 2019
Motorway Road Markings in Whitestone #Motorway #Marking #Whitestone https://t.co/q0O0uq1IUQ
— Marking Contractors (@markingcontract) February 25, 2019
Professor DV Bishop outlines the multiple flaws in the TEF methodology
In a previous post I questioned the rationale and validity of the Teaching Excellence and Student Outcomes Framework (TEF). Here I document the technical and statistical problems with TEF.
Two types of data are combined in TEF: a set of ‘contextual’ variables, including student backgrounds, subject of study, level of disadvantage, etc., and a set of ‘quality indicators’ as follows:
As detailed further below, data on the institution’s quality indicators is compared with the ‘expected value’ that is computed based on the contextual data of the institution. Discrepancies between obtained and expected values, either positive or negative, are flagged and used, together with a written narrative from the institution, to rate each institution as Gold, Silver or Bronze. This beginner’s guide provides more information.
When you visit the DfE’s website, the first impression is that it is a model of transparency. On this site, you can download tables of data and even consult interactive workbooks that allow you to see the relevant statistics for a given provider. Track through the maze of links and you can also find an 87-page technical document of astounding complexity that specifies the algorithms used to derive the indicators from the underlying student data, DLHE survey and NSS data.
The problem, however, is that nowhere can you find a script that documents the process of deriving the final set of indicators from the raw data: if you try to work this out from first principles by following the HESA guidance on benchmarking, you run into the sand, because the institutional data is not provided in the right format. When I asked the TEF metrics team about this, I was told: “The full process from the raw data in HESA/ILR returns, NSS etc. cannot be made fully open due to data protection issues, as there is sensitive student information involved in the process.” But this seems disingenuous. I can see that student data files are confidential, but once this information has been extracted and aggregated at institutional level, it should be possible to share it. If that isn’t feasible, then the metrics team should be able to at least generate some dummy data sets, with scripts that would do the computations that convert the raw metrics into the flags that are used in TEF rankings.
As someone interested in reproducibility in science, I’m all too well aware of the problems that can ensue if the pipeline from raw data to results is not clearly documented – this short piece by Florian Markowetz makes the case nicely. In science and beyond, there are some classic scare stories of what can happen when the analysis relies on spreadsheets: there’s even a European Spreadsheet Risks Interest Group. There will always be errors in data – and sometimes also in the analysis scripts: the best way to find and eradicate them is to make everything open.
The idea of benchmarking is to avoid penalising institutions that take on students from disadvantaged backgrounds:
“Through benchmarking, the TEF metrics take into account the entry qualifications and characteristics of students, and the subjects studied, at each university or college. These can be very different and TEF assessment is based on what each college or university achieves for its particular students within this context. The metrics are also considered alongside further contextual data, about student characteristics at the provider as well as the provider’s location and provision.”
One danger of benchmarking is that it risks entrenching disadvantage. Suppose we have institutions X and Y, which are polar opposites in terms of how well they treat students. X is only interested in getting student fees, does not teach properly, and does not care about drop-outs – we hope such cases are rare, but, as this Panorama exposé showed, they do exist, and we’d hope that TEF would expose them. Y, by contrast, fosters its students and does everything possible to ensure they complete their course. Let us further suppose that X offers a limited range of vocational courses, whereas Y offers a wider range of academic subjects, and that X has a higher proportion of disadvantaged students. Benchmarking ensures that X will be evaluated relative to other institutions offering similar courses to a similar population. This can lead to a situation where, because poor outcomes at X are correlated with its subject and student profile, expectations are low, and poor scores for student satisfaction and completion rates are not penalised.
Benchmarking is well-intentioned – its aim is to give institutions a chance to shine even if they are working with students who may struggle to learn. However, it runs the risk of making low expectations acceptable. It could be argued that, while there are characteristics of students and courses that affect student outcomes, in general, higher education institutions should not be offering courses where there is a high probability of student drop-out. And students would find it more helpful to see raw data on drop-out rates and student satisfaction, than to merely be told that an institution is Bronze, Silver or Gold – a rating that can only be understood in relative terms.
The method used to do benchmarking comes from Draper and Gittoes (2005), and is explained here. A more comprehensive statistical treatment and critique can be found here. Essentially, you identify background variables that predict outcomes, assess typical outcomes associated with each combination of these in the whole population under consideration, and then calculate an ‘expected’ score, as a mean of these combinations, weighted by the frequency of each combination at the institution.
The obtained score may be higher or lower than the ‘expected’ value. The question is how you interpret such differences, bearing in mind that some variation is expected just due to random fluctuations. The precision of the estimate of both observed and expected values will increase as the sample size increases: you can compute a standard error around the difference score, and then use statistical criteria to identify cases with difference scores that are likely to be meaningful and not just down to random noise. However, where there is a small number of students, it is hard to distinguish a genuine effect from noise, but where there is a very large number, even tiny differences will be significant. The process used in benchmarking uses statistical criteria to assign ‘flags’ to indicate scores that are extremely good (++), or good (+), or extremely bad (–) or bad (-) in relation to expectation. To ameliorate the problem of tiny effects being flagged in large samples, departures from expectation are flagged only if they exceed a specific number of percentage points.
This is illustrated for the case of one of the NSS measurements in Figure 1, which shows that the problem of sample size has not been solved: a large institution is far more likely to get a flagged score (either positive or negative) than a small one. Indeed, a small institution is a pretty safe bet for a silver award.
Figure 1. The Indicator (x-axis) is the percentage of students with positive NSS ratings, and the z-score (y-axis) shows how far this value is from expectation based on benchmarks. The plot illustrates several things: (a) the range of indicators becomes narrower as sample size increases; (b) most scores are bunched around 85%; © for large institutions, even small changes in indicators can make a big difference to flags, whereas for small institutions, most are unflagged, regardless of the level of indicator; (d) the number of extreme flags (filled circles or asterisks) is far greater for large than small institutions.
From a student perspective, it is crucial to have information about specific courses; institution-wide evaluation is not much use to anyone other than vice-chancellors who wish to brag about their rating. However, the problems I have outlined with small samples are amplified if we move to subject-level evaluation. I raised this issue with the TEF metrics team, and was told:
‘The issue of smaller student numbers ‘defaulting’ to silver is something we are aware of. Paragraph 94 on page 29 of the report on findings from the first subject pilot mentions some OfS analysis on this. The Government consultation response also has a section on this. On page 40, the government response to question 10 refers to assessability, and potential methods that could be used to deal with this in future runs of the TEF.’
So the OfS knows they have a problem, but seems determined to press on, rather than rethinking the exercise.
The benchmarks used in TEF are based on identifying statistical outliers. Forget for a moment the sample size issue, and suppose we have a set of institutions with broadly the same large number of students, and a spread of scores on a metric, such that the mean percentage meeting criterion is 80%, with a standard deviation of 2% (see Figure 2). We flag the bottom 10% (those with scores below 77.5%) as problematic. In the next iteration of the exercise, those with low scores have either gone out of business, improved their performance, or learned how to game the metric, and so we no longer have anyone scoring below 77.5%. The mean score thus increases and the standard error decreases. So now, on statistical grounds, a score below 78.1% gets flagged as problematic. In short, with a statistical criterion for poor performance, even if everyone improves dramatically, or poor-performers drop out, there will still be those at the bottom of the distribution – unless we get to a point where there is no meaningful variation in scores.
Figure 2: Simulated data showing how improvements in scores can lead to increasing cutoff in the next round if statistical criterion is adopted.
TEF may be summarised thus:
All to end up with a three-point ordinal scale, which does not provide students with the information that they need to select a course.
Time, maybe, to ditch the TEF and encourage students to consult the raw data instead to find out about courses?
Road Relining Contractors in Argoed #Roadway #Remarking #Experts #Argoed https://t.co/VZzg2XJSM0
— Marking Contractors (@markingcontract) February 25, 2019
Line Marking Contractors in The Rampings #The #Rampings https://t.co/cuTY2UrBi0
— Marking Contractors (@markingcontract) February 25, 2019
Road Line Marking in Duntisbourne Leer #Roadway #Line #Markings #Duntisbourne #Leer https://t.co/eKqb8I4m0R
— Marking Contractors (@markingcontract) February 24, 2019
Motorway Road Markings in Birchall #Motorway #Marking #Birchall https://t.co/9SRZGRTAYy
— Marking Contractors (@markingcontract) February 24, 2019
Thermoplastic Playground Markings in North Down #School #Playground #Marking #North #Down https://t.co/BvUoE7GABq
— Marking Contractors (@markingcontract) February 24, 2019
Car Park Repainting in Barrachnie #Car #Parking #Bay #Relining #Barrachnie https://t.co/OiDTGkGhPJ
— Marking Contractors (@markingcontract) February 24, 2019
Thermoplastic Playground Markings in Little Fenton #School #Playground #Marking #Little #Fenton https://t.co/7RtxILt64m
— Marking Contractors (@markingcontract) February 24, 2019
Road Relining Contractors in Lisburn #Roadway #Remarking #Experts #Lisburn https://t.co/Kb2xxHviPV
— Marking Contractors (@markingcontract) February 23, 2019
Road Markings Meanings in Atcham #Road #Line #Meanings #Atcham https://t.co/0QLQToyW5I
— Marking Contractors (@markingcontract) February 23, 2019
Car Park Painting in Barmby Moor #Parking #Bay #Colour #Coating #Barmby #Moor https://t.co/gU2xNYGTrj
— Marking Contractors (@markingcontract) February 23, 2019
Car Park Lining Services in Clackmannanshire #Car #Park #Line #Marking #Clackmannanshire https://t.co/WmtJjKlgoE
— Marking Contractors (@markingcontract) February 23, 2019
Car Park Repainting in Trevenen Bal #Car #Parking #Bay #Relining #Trevenen #Bal https://t.co/nl7MELvjqH
— Marking Contractors (@markingcontract) February 22, 2019
Road Line Marking in Magherafelt #Roadway #Line #Markings #Magherafelt https://t.co/HwzNn0PahG
— Marking Contractors (@markingcontract) February 22, 2019
Car Park Painting in Hertfordshire #Parking #Bay #Colour #Coating #Hertfordshire https://t.co/4TnMK4qfiB
— Marking Contractors (@markingcontract) February 22, 2019
Warehouse Floor Paint in Caerphilly #Warehouse #Surface #Painting #Caerphilly https://t.co/6SY4E7GmYv
— Marking Contractors (@markingcontract) February 22, 2019
Car Park Painting in Carnegie #Parking #Bay #Colour #Coating #Carnegie https://t.co/PCw58h2Vx0
— Marking Contractors (@markingcontract) February 22, 2019
Thermoplastic Road Marking in Maenaddwyn #Thermo #Plastic #Roadway #Markings #Maenaddwyn https://t.co/CA68tExG95
— Marking Contractors (@markingcontract) February 21, 2019
Warehouse Floor Paint in Bolnhurst #Warehouse #Surface #Painting #Bolnhurst https://t.co/rxBFcPaQlj
— Marking Contractors (@markingcontract) February 21, 2019
Road Line Marking in Stanley #Roadway #Line #Markings #Stanley https://t.co/pZdimA42iM
— Marking Contractors (@markingcontract) February 21, 2019
Warehouse Walkway Markings in East Stanley #Factory #Footpath #Marking #East #Stanley https://t.co/a7uqiqhZkb
— Marking Contractors (@markingcontract) February 21, 2019
Road Markings Meanings in Midlothian #Road #Line #Meanings #Midlothian https://t.co/tjI0vpGI5D
— Marking Contractors (@markingcontract) February 20, 2019
Car Park Markings in Findochty #Carpark #Line #Marking #Findochty https://t.co/mcAyTfWpgr
— Marking Contractors (@markingcontract) February 20, 2019
Car Park Lining Services in Wern-Gifford #Car #Park #Line #Marking #Wern-Gifford https://t.co/4iXvJDkFgF
— Marking Contractors (@markingcontract) February 20, 2019
Thermoplastic Playground Markings in Hampshire #School #Playground #Marking #Hampshire https://t.co/OZ92yZOxmH
— Marking Contractors (@markingcontract) February 20, 2019
Motorway Road Markings in East Riding of Yorkshire #Motorway #Marking #East #Riding #of #Yorkshire https://t.co/CQocRX0jxL
— Marking Contractors (@markingcontract) February 20, 2019
Road Markings Meanings in Castleton #Road #Line #Meanings #Castleton https://t.co/mhbOPH9Xw9
— Marking Contractors (@markingcontract) February 19, 2019
Some highlights from a decade of research with mice
from RSSMix.com Mix ID 8239600 http://www.understandinganimalresearch.org.uk/news/communications-media/ten-years-of-research-with-mice/
via IFTTT
High Friction Surfacing in Hampton Green #Anti #Skid #Surfacing #Hampton #Green https://t.co/Y4BjoDfdG9
— Marking Contractors (@markingcontract) February 19, 2019
Car Park Markings in Ballinger Common #Carpark #Line #Marking #Ballinger #Common https://t.co/2tDuBMuZDa
— Marking Contractors (@markingcontract) February 19, 2019
High Friction Surfacing in Ashlett #Anti #Skid #Surfacing #Ashlett https://t.co/W96k0KOUnO
— Marking Contractors (@markingcontract) February 19, 2019
High Friction Surfacing in Down #Anti #Skid #Surfacing #Down https://t.co/cKm2kVCgZO
— Marking Contractors (@markingcontract) February 18, 2019
