Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An algorithm that blew up Italy’s school system

algorithmwatch.org

Post reply on HN
Print view View original

11–20 of 52 posts

Re: An algorithm that blew up Italy’s school system

#11

Post by satellite2 »

jaclaz wrote: post #7
st-keller wrote: post #3

Why do i read such stories at all? Now i can‘t tell, if it’s the algorithm or the implementation or both and i’m left with more questions than before! Edit: typo

Seemingly it is implementation: >Why did such errors occur? >When the algorithm finds an ideal candidate for a position, it does not reset the list of remaining candidates before commencing the search to fill the next vacancy. Thus, those candidates who missed out on the first role that matched their preferences are definitively discarded from the pool of available teachers, with no possibility of employment. The alg…

Yes that's the explanation according to the article. But that seems sketchy. With such an algorithm the pool of potential candidates would quickly converge to 0. So I doubt the algorithm exactly does this.

Re: An algorithm that blew up Italy’s school system

#12

Post by ghaff »

morkalork wrote: post #8
st-keller wrote: post #3

Why do i read such stories at all? Now i can‘t tell, if it’s the algorithm or the implementation or both and i’m left with more questions than before! Edit: typo

It could serve as a parable you'd tell a junior engineer interested writing algorithms that get applied to people's livelihood. Beware your own hubris, demonstrate or simulate results and show them to all stakeholders like the union, do controlled rollouts (why not start with 1 region in the country?), verify inputs that could be untrustworthy. Basically all the usual lessons one learns the first time they write an a…

>demonstrate or simulate results and show them to all stakeholders

This is a very important point. Especially for any algorithm/policy change that affects people's pay, time off, scheduling, etc., it may seem like a reasonable change to you and to management but a lot of people will have a visceral reaction anyway. Not everyone may be happy at the end but it's important they're at least part of the process.

Re: An algorithm that blew up Italy’s school system

#13

Post by b800h »

What is infuriating about this is that this should be a solved optimisation problem, but it has probably been outsourced on the cheap to some bog-standard web dev team who have tried to figure it out from first principles. It's a bit like trying to roll your own security.

Re: An algorithm that blew up Italy’s school system

#14

Post by duskwuff »

escanor wrote: post #5

Who does that algorithm really serve?

From the sounds of it: no one, it's just bad. Teachers are passed over for jobs that they should be getting, schools are having difficulty filling teaching roles (and are probably paying extra for temporary teachers), and students are being instructed poorly by a revolving door of poorly qualified teachers. This isn't a zero-sum scenario; getting the right teachers into the right jobs makes everyone happier.

Re: An algorithm that blew up Italy’s school system

#15

Post by duskwuff »

satellite2 wrote: post #11
jaclaz wrote: post #7

Earlier quoted context omitted.

Seemingly it is implementation: >Why did such errors occur? >When the algorithm finds an ideal candidate for a position, it does not reset the list of remaining candidates before commencing the search to fill the next vacancy. Thus, those candidates who missed out on the first role that matched their preferences are definitively discarded from the pool of available teachers, with no possibility of employment. The alg…

Yes that's the explanation according to the article. But that seems sketchy. With such an algorithm the pool of potential candidates would quickly converge to 0. So I doubt the algorithm exactly does this.

> With such an algorithm the pool of potential candidates would quickly converge to 0.

From the sound of things, that may have been happening:

> Moreover, recruitment is not keeping pace with schools' need for teachers, thus leaving hundreds of classroom positions vacant which is affecting the educational progress of students all over Italy.

Re: An algorithm that blew up Italy’s school system

#16

Post by basseq »

jaclaz wrote: post #7
st-keller wrote: post #3

Why do i read such stories at all? Now i can‘t tell, if it’s the algorithm or the implementation or both and i’m left with more questions than before! Edit: typo

Seemingly it is implementation: >Why did such errors occur? >When the algorithm finds an ideal candidate for a position, it does not reset the list of remaining candidates before commencing the search to fill the next vacancy. Thus, those candidates who missed out on the first role that matched their preferences are definitively discarded from the pool of available teachers, with no possibility of employment. The alg…

That "drop out" concept just seems entirely wrong. Not only because its erroneously constricting the applicant pool, but because it strongly biases the earlier roles in the queue.

For example, consider two roles and two applicants, with fit scores as below:

               Role 1    Role 2
  -----        -----     -----
  Applicant A    96%       95%
  Applicant B    95%       50%
Ignoring the "drop out" bug, under the algorithm described the system would evaluate all candidates for Role 1, determine Applicant A is the best, then move on. At that point, Applicant B is the best candidate for Role 2... even though they're not a very good one. Overall, not a great outcome (73% avg.).

You'd think the algorithm would want to maximize outcomes across all roles: the more optimal "best fit" solution would be Applicant B in Role 1 and Applicant A in Role 2 (95% avg).

(I'm assuming the reality here is that Role B isn't available at time of evaluation, so there's no way to evaluate the universe without waiting, which may be sub-optimal.)

Re: An algorithm that blew up Italy’s school system

#17

Post by rapsey »

b800h wrote: post #13

What is infuriating about this is that this should be a solved optimisation problem, but it has probably been outsourced on the cheap to some bog-standard web dev team who have tried to figure it out from first principles. It's a bit like trying to roll your own security.

Nah this is a government contract. It was definitely done by a consultancy for big money and zero effort and no accountability.

Re: An algorithm that blew up Italy’s school system

#18

Post by bezout »

The way it works in Italy: the design and implementation of the algorithm in Java 8 or an even older version of C# was probably outsourced to one of the Big 4 consultancy companies.

A group of underpaid graduates was put together to crack the problem. All of them crammed for their algo & ds exam since that’s what the Italian university system incentivises so none of them did actually remember a thing about algorithm design. They googled a bunch of words and forked the first PoC they found on GitHub.

Everything was wrapped into a nice PowerPoint full of corporate BS and delivered to the government.

Edit: As expected, the algorithm was developed by a company owned by Dxc Technology [1] and Leonardo which is the Italian defence company. The contract was worth 5 million of euros.

[1]: https://www.wired.it/article/algoritmo-scuola-supplenze-mini...

Re: An algorithm that blew up Italy’s school system

#19

Post by [deleted] »

rapsey wrote: post #17
b800h wrote: post #13

What is infuriating about this is that this should be a solved optimisation problem, but it has probably been outsourced on the cheap to some bog-standard web dev team who have tried to figure it out from first principles. It's a bit like trying to roll your own security.

Nah this is a government contract. It was definitely done by a consultancy for big money and zero effort and no accountability.

[deleted]

Re: An algorithm that blew up Italy’s school system

#20

Post by lordnacho »

b800h wrote: post #13

What is infuriating about this is that this should be a solved optimisation problem, but it has probably been outsourced on the cheap to some bog-standard web dev team who have tried to figure it out from first principles. It's a bit like trying to roll your own security.

What I don't get is why it would be outsourced at all. Run a team that manages the entire SDLC. If they mess it up, they are maintaining it and have a reason to solve the problem.

How did people manage to convince government that software is this one-time thing you just pay for and then it works forever?

Post reply on HN