UK Post Office Scandal

[Blackmac]

So the potential bill for this is £900 million. FFS. Public bloody money.

[Sandstorm]

So the potential bill for this is £900 million. FFS. Public bloody money.

Get half from those dicks at Fujitsu.

[fishfoodie]

So the potential bill for this is £900 million. FFS. Public bloody money.

Get half from those dicks at Fujitsu.

Well as a sometimes Developer, I feel duty bound to point out that:

(a) You generally get what you pay for.
(b) The Specs you write, are what the Developers are paid to produce
(c) The people writing the Specs are never, ever, the poor slobs who will use the product.
(d) If the end product meets the Specs, you don't get to bitch about the outcome.
(e) The Customer is the one doing the validation, so if they don't spot any legitimate issues, & a legitimate issue is only not meeting the Specs, that's your fucking problem !
(f) If you let the vendor tell you the product is done, you're a fucking eejiit !

In summary; if I were to choose between guessing whether the problems lay with the politically appointed management in the PO, & the multi-national that has successfully garnered many IT projects,& continues to do so, my money is on the muppets in the PO making a complete balls of the job

[Random1]

So the potential bill for this is £900 million. FFS. Public bloody money.

Get half from those dicks at Fujitsu.

Well as a sometimes Developer, I feel duty bound to point out that:

(a) You generally get what you pay for.
(b) The Specs you write, are what the Developers are paid to produce
(c) The people writing the Specs are never, ever, the poor slobs who will use the product.
(d) If the end product meets the Specs, you don't get to bitch about the outcome.
(e) The Customer is the one doing the validation, so if they don't spot any legitimate issues, & a legitimate issue is only not meeting the Specs, that's your fucking problem !
(f) If you let the vendor tell you the product is done, you're a fucking eejiit !

In summary; if I were to choose between guessing whether the problems lay with the politically appointed management in the PO, & the multi-national that has successfully garnered many IT projects,& continues to do so, my money is on the muppets in the PO making a complete balls of the job

A lot of property builders/developers take this approach.

That’s how we end up with grenfell etc.

Not sure this feels any different to me. Is the comparison flawed?

[Rhubarb & Custard]

I suspect in this case we'll find ill-defined (and also quite possibly changing) requirements and some poor work in development being done by people who didn't know the field.

Overall the PO takes responsibility as they're the ones buying the product. Also critical is what were the lines of communication between supplier and purchaser, operationally how did that work, who was signing off...

And we also need to know what was signed off to go live, almost certainly it was signed off with flaws, perhaps some that weren't known as the volume testing of live was never matched in testing (and some not known if the testing was for shit), but clearly they knew there were some problems, so what did both parties agree was actually going into a production/live environment?

[fishfoodie]

Get half from those dicks at Fujitsu.

Well as a sometimes Developer, I feel duty bound to point out that:

(a) You generally get what you pay for.
(b) The Specs you write, are what the Developers are paid to produce
(c) The people writing the Specs are never, ever, the poor slobs who will use the product.
(d) If the end product meets the Specs, you don't get to bitch about the outcome.
(e) The Customer is the one doing the validation, so if they don't spot any legitimate issues, & a legitimate issue is only not meeting the Specs, that's your fucking problem !
(f) If you let the vendor tell you the product is done, you're a fucking eejiit !

In summary; if I were to choose between guessing whether the problems lay with the politically appointed management in the PO, & the multi-national that has successfully garnered many IT projects,& continues to do so, my money is on the muppets in the PO making a complete balls of the job

A lot of property builders/developers take this approach.

That’s how we end up with grenfell etc.

Not sure this feels any different to me. Is the comparison flawed?

In Construction you have Standards, & Engineers understand the need, at the bare minimum, to meet those, regardless of the job specification.

For Grenfell you had a situation where someone was specifying materials that weren't approved for the use they were being used in; e.g. the insulation was certified for floors < 3, but they specified it all the way up.

You might see "Engineer" a lot in IT jobs, but it's an ongoing source of aggravation, because IT is so immature compared to Engineering that there is no equivalence in terms of Standards & Certification & as such; while you might have an expectation that a Civil Engineer can build you a bridge that won't fall collapse when the first car rolls over it, best of luck when you select a Cloud Security Engineer tells you your data is safe !

[fishfoodie]

I suspect in this case we'll find ill-defined (and also quite possibly changing) requirements and some poor work in development being done by people who didn't know the field.

Overall the PO takes responsibility as they're the ones buying the product. Also critical is what were the lines of communication between supplier and purchaser, operationally how did that work, who was signing off...

And we also need to know what was signed off to go live, almost certainly it was signed off with flaws, perhaps some that weren't known as the volume testing of live was never matched in testing (and some not known if the testing was for shit), but clearly they knew there were some problems, so what did both parties agree was actually going into a production/live environment?

My general experience is that the budget is the ultimate decider of the end product; so I expect that the PO decided on a budget, & then the Development took "x", & they allowed "z" overall, & had "y" for bug fixing etc, & when "x" & "y" were exhausted, they called it a day & shipped the steaming pile of shite regardless of how badly it worked.

What might have also realistically also happened is that "x" + "y" were exhausted before they'd even got to the MVP, & they poured in more money until the pips squeaked, & eventually someone in the PO said; "Ship It !"

[Sandstorm]

Fujitsu supported Horizen throught the years these poor people were getting shafted and never once owned up to their software being flawed. Fuck ‘em.

[Rhubarb & Custard]

Oftentimes a serious vendor can tell your, pretty closely, that x requirements costs y development time which is to say cost. But often they themselves would have some idea what they're selling.

This I suspect was a little different being a venture into the unknown for both sides, and likely being led on the PO side by some pig ignorant folks. Fujitsu would have known software, but very possibly not the business they were selling into, and that's a bigger problem again if as I suspect nobody on the PO side actually understood what they wanted, the just wanted something that worked and was cheaper.

I've been in a position before with a well known financial provider who'd moved their ISA holdings onto a new system and was out of contract with their old system vendor, and they wanted me to sign on a Thursday else they'd have to stop trading on the ISA accounts come Monday morning and the system clearly not only didn't work itself, it also didn't meet some business specific requirements. ISA retail accounts were hardly the biggest part of their product offerings, but it certainly would have been noticed. Quite how they made it though various audits I've no idea, but it can't be good, either incompetence on a wondrous scale, or worse. (I wasn't by any means the person who should have been signing, clearly they'd worked a long way down the chain trying to find someone who'd sign). But for sure these things can get pushed even by supposed serious businesses. And this one was about cost, both deciding not to stay with the existing vendor and going with a cheaper new offering, and then getting forced if not quite at gunpoint to consider they'd be operating without a contract. The new system did go live, not with me signing, someone did, but they only signed for and on behalf of, and I was told they even signed in pencil, and I heard that from a number involved.

[Rhubarb & Custard]

Fujitsu supported Horizen throught the years these poor people were getting shafted and never once owned up to their software being flawed. Fuck ‘em.

They did, but it's also possible they met the terms of what the Post Office had asked of them. That's very much something which needs to be looked into, sadly there could very easily be nobody competent involved in the looking into it part.

[inactionman]

I suspect in this case we'll find ill-defined (and also quite possibly changing) requirements and some poor work in development being done by people who didn't know the field.

Overall the PO takes responsibility as they're the ones buying the product. Also critical is what were the lines of communication between supplier and purchaser, operationally how did that work, who was signing off...

And we also need to know what was signed off to go live, almost certainly it was signed off with flaws, perhaps some that weren't known as the volume testing of live was never matched in testing (and some not known if the testing was for shit), but clearly they knew there were some problems, so what did both parties agree was actually going into a production/live environment?

From what I've seen of this, it just looks like bad coding by a large consultancy that put too many inexperienced (read: cheap) people into delivery and didn't support them properly.

As just one example of countless, the system would double-account transactions if it hung or otherwise crashed upon submission, without telling the user or allowing for correction. This is very unlikely to be a stated user requirement (to be clear, that it needs to be stated that the system needs avoid this situation), it should be a development standard to be able to manage nonrepudiation, message delivery and sequencing, and recovery of service outage, assuming that the underlying issue lies somewhere in that lot.

The user requirement would be to allow a sequence of debits and credits to be entered and for the system to process these, presumably in order of entry - the user would, never expect to say 'oh and don't fucking add the same transaction twice if the shoddy system falls over and forgets where it got to'.

Even despite this, the project team itself should elicit any technical requirements- e.g. recovery point objectives which say how far back the system needs to recover if it fails - the business user would have no clue whatsoever that this even needs to be thought about. The project team should perform all the unit and system tests to make sure it functions as cogent system, including performance and recovery - although this would be done by a dedicated test team, this team is generally a part of the project (security would be tested but the most stringent of that would be done for the most part by a third party, to avoid the project team marking too much of their own homework). The end user performs a User Acceptance Test that just warrant the system performs as required as part of the business process - they might do some negative tests e.g. try to log the same user on twice at the same time, deliberately mis-enter account numbers etc, and they may indeed pick up some of the issues that horizon suffered from, but these should be caught elsewhere. They certainly won't be expecting to test for technical fuck-ups of the type I've seen described.

In short, the main testing done by the end customer won't cover the situations that seem to have occurred which have caused the problems. Bog-standard good development practice should have prevented it, and it's not entirely unreasonable for the customer to expect the delivery partner to be actively managing this.

I appreciate the development team and testing team may have flagged many of these issues, but it seems they did not actually specify the criticality or the actual impact of these very clearly. I also appreciate this was all 20 years or more ago, but these are very basic practices.

Because, of course if they admit to shit development practices and intrinsic flaws then they ain;t getting paid.

I've suffered this far too many times in my professional life, hence my slightly shouty response. It's pretty much par for the course with many of these big consultancy and delivery practices - win the job, put the cheap grads on and blag/brazen the delivery, thus maximising profit and bonuses even if the customer does not get what they expected.


As a weak analogy, if I had an extension built for my house I'd check the room dimensions are as specified and do some basic checks that windows don't leak etc, but I'm not going to assess whether the wiring may lead to a fire risk over time or whether the foundations are likely to sag. The developer should be actively doing that, as it's their day job to know about it and the nuances of the build. Even if the developer reported technical issues to me, it's on them to tell me that 'this is a show-stopper' rather than let my house burn down or collapse in on itself. If the room sizes aren't right for my family, that would be my responsibility (or the architect's), not the developer's. That's where responsibilities lie.

[Sandstorm]

Tl;dr (from both of you)

Short answer: Fujitsu were shit, PO incompetent. Both covered it up and should be shot today.

[Sandstorm]

.

[inactionman]

Tl;dr (from both of you)

Short answer: Fujitsu were shit, PO incompetent. Both covered it up and should be shot today.

Always surprised people admit to being lazy. It's hardly War and Peace.

And the PO won't be the first - or last - organisation to be browbeaten or otherwise gaslighted by consultancies into accepting software that doesn't do what they thought was agreed.

(This is not to excuse the dickheads who refused to accept the software was shite and hounded sub post masters - that is the PO's real offence in all this. My posts are about why the system was shit in the first instance).

[Rhubarb & Custard]

I suspect in this case we'll find ill-defined (and also quite possibly changing) requirements and some poor work in development being done by people who didn't know the field.

Overall the PO takes responsibility as they're the ones buying the product. Also critical is what were the lines of communication between supplier and purchaser, operationally how did that work, who was signing off...

And we also need to know what was signed off to go live, almost certainly it was signed off with flaws, perhaps some that weren't known as the volume testing of live was never matched in testing (and some not known if the testing was for shit), but clearly they knew there were some problems, so what did both parties agree was actually going into a production/live environment?

From what I've seen of this, it just looks like bad coding by a large consultancy that put too many inexperienced (read: cheap) people into delivery and didn't support them properly.

As just one example of countless, the system would double-account transactions if it hung or otherwise crashed upon submission, without telling the user or allowing for correction. This is very unlikely to be a stated user requirement (to be clear, that it needs to be stated that the system needs avoid this situation), it should be a development standard to be able to manage nonrepudiation, message delivery and sequencing, and recovery of service outage, assuming that the underlying issue lies somewhere in that lot.

The user requirement would be to allow a sequence of debits and credits to be entered and for the system to process these, presumably in order of entry - the user would, never expect to say 'oh and don't fucking add the same transaction twice if the shoddy system falls over and forgets where it got to'.

Even despite this, the project team itself should elicit any technical requirements- e.g. recovery point objectives which say how far back the system needs to recover if it fails - the business user would have no clue whatsoever that this even needs to be thought about. The project team should perform all the unit and system tests to make sure it functions as cogent system, including performance and recovery - although this would be done by a dedicated test team, this team is generally a part of the project (security would be tested but the most stringent of that would be done for the most part by a third party, to avoid the project team marking too much of their own homework). The end user performs a User Acceptance Test that just warrant the system performs as required as part of the business process - they might do some negative tests e.g. try to log the same user on twice at the same time, deliberately mis-enter account numbers etc, and they may indeed pick up some of the issues that horizon suffered from, but these should be caught elsewhere. They certainly won't be expecting to test for technical fuck-ups of the type I've seen described.

In short, the main testing done by the end customer won't cover the situations that seem to have occurred which have caused the problems. Bog-standard good development practice should have prevented it, and it's not entirely unreasonable for the customer to expect the delivery partner to be actively managing this.

I appreciate the development team and testing team may have flagged many of these issues, but it seems they did not actually specify the criticality or the actual impact of these very clearly. I also appreciate this was all 20 years or more ago, but these are very basic practices.

Because, of course if they admit to shit development practices and intrinsic flaws then they ain;t getting paid.

I've suffered this far too many times in my professional life, hence my slightly shouty response. It's pretty much par for the course with many of these big consultancy and delivery practices - win the job, put the cheap grads on and blag/brazen the delivery, thus maximising profit and bonuses even if the customer does not get what they expected.


As a weak analogy, if I had an extension built for my house I'd check the room dimensions are as specified and do some basic checks that windows don't leak etc, but I'm not going to assess whether the wiring may lead to a fire risk over time or whether the foundations are likely to sag. The developer should be actively doing that, as it's their day job to know about it and the nuances of the build. Even if the developer reported technical issues to me, it's on them to tell me that 'this is a show-stopper' rather than let my house burn down or collapse in on itself. If the room sizes aren't right for my family, that would be my responsibility (or the architect's), not the developer's. That's where responsibilities lie.

Quite possibly.

But we still need to know what the communication operations processes were between the PO and their supplier before during and after the project, we need to know what was defined as the requirements, what the testing detailed, and what was signed off as going live.

And because the PO own the business I'm putting them first in the accountability stakes, at least for now, but I dislike the notion when you outsource work your outsource risk/responsibility. It's possible the PO could convince me they did everything that could reasonably be expected as due diligence on their part, but the impression they're giving doesn't support that. I'm much more inclined to think we had a bunch of people with a bad grasp of Classics, Theology and PPE running the implementation with little to no understanding of what they were doing, or at least asking for, and ignorance is a piss poor defence.

And of course the whole thing will be muddied further by the respective involvements of the Japanese and UK government.

[inactionman]

I suspect in this case we'll find ill-defined (and also quite possibly changing) requirements and some poor work in development being done by people who didn't know the field.

Overall the PO takes responsibility as they're the ones buying the product. Also critical is what were the lines of communication between supplier and purchaser, operationally how did that work, who was signing off...

And we also need to know what was signed off to go live, almost certainly it was signed off with flaws, perhaps some that weren't known as the volume testing of live was never matched in testing (and some not known if the testing was for shit), but clearly they knew there were some problems, so what did both parties agree was actually going into a production/live environment?

From what I've seen of this, it just looks like bad coding by a large consultancy that put too many inexperienced (read: cheap) people into delivery and didn't support them properly.

As just one example of countless, the system would double-account transactions if it hung or otherwise crashed upon submission, without telling the user or allowing for correction. This is very unlikely to be a stated user requirement (to be clear, that it needs to be stated that the system needs avoid this situation), it should be a development standard to be able to manage nonrepudiation, message delivery and sequencing, and recovery of service outage, assuming that the underlying issue lies somewhere in that lot.

The user requirement would be to allow a sequence of debits and credits to be entered and for the system to process these, presumably in order of entry - the user would, never expect to say 'oh and don't fucking add the same transaction twice if the shoddy system falls over and forgets where it got to'.

Even despite this, the project team itself should elicit any technical requirements- e.g. recovery point objectives which say how far back the system needs to recover if it fails - the business user would have no clue whatsoever that this even needs to be thought about. The project team should perform all the unit and system tests to make sure it functions as cogent system, including performance and recovery - although this would be done by a dedicated test team, this team is generally a part of the project (security would be tested but the most stringent of that would be done for the most part by a third party, to avoid the project team marking too much of their own homework). The end user performs a User Acceptance Test that just warrant the system performs as required as part of the business process - they might do some negative tests e.g. try to log the same user on twice at the same time, deliberately mis-enter account numbers etc, and they may indeed pick up some of the issues that horizon suffered from, but these should be caught elsewhere. They certainly won't be expecting to test for technical fuck-ups of the type I've seen described.

In short, the main testing done by the end customer won't cover the situations that seem to have occurred which have caused the problems. Bog-standard good development practice should have prevented it, and it's not entirely unreasonable for the customer to expect the delivery partner to be actively managing this.

I appreciate the development team and testing team may have flagged many of these issues, but it seems they did not actually specify the criticality or the actual impact of these very clearly. I also appreciate this was all 20 years or more ago, but these are very basic practices.

Because, of course if they admit to shit development practices and intrinsic flaws then they ain;t getting paid.

I've suffered this far too many times in my professional life, hence my slightly shouty response. It's pretty much par for the course with many of these big consultancy and delivery practices - win the job, put the cheap grads on and blag/brazen the delivery, thus maximising profit and bonuses even if the customer does not get what they expected.


As a weak analogy, if I had an extension built for my house I'd check the room dimensions are as specified and do some basic checks that windows don't leak etc, but I'm not going to assess whether the wiring may lead to a fire risk over time or whether the foundations are likely to sag. The developer should be actively doing that, as it's their day job to know about it and the nuances of the build. Even if the developer reported technical issues to me, it's on them to tell me that 'this is a show-stopper' rather than let my house burn down or collapse in on itself. If the room sizes aren't right for my family, that would be my responsibility (or the architect's), not the developer's. That's where responsibilities lie.

Quite possibly.

But we still need to know what the communication operations processes were between the PO and their supplier before during and after the project, we need to know what was defined as the requirements, what the testing detailed, and what was signed off as going live.

And because the PO own the business I'm putting them first in the accountability stakes, at least for now, but I dislike the notion when you outsource work your outsource risk/responsibility. It's possible the PO could convince me they did everything that could reasonably be expected as due diligence on their part, but the impression they're giving doesn't support that. I'm much more inclined to think we had a bunch of people with a bad grasp of Classics, Theology and PPE running the implementation with little to no understanding of what they were doing, or at least asking for, and ignorance is a piss poor defence.

And of course the whole thing will be muddied further by the respective involvements of the Japanese and UK government.

I agree you don't/can't outsource all risk, but you certainly would expect the supplier to assume the responsibility of developing a coherent system that can manage very specific technical faults. A server losing contact with client (if that is the architecture) should not result in invisible transactions being added. It just shouldn't.

If the PO bought a fleet of trucks they'd assume responsibility that the trucks can take the loads safely and can be safely and effectively operated on UK roads, but they can't assume the risk of the engines exploding as the manufacturer got their metallurgy wrong*. In a number of cases it's that level of failure from Fujitsu, and would not be obvious test failures as some that I've seen documented would only occurred under a very specific set of conditions that couldn't be guaranteed to be be replicated under final acceptance test. They should have been picked up under unit test where that sort of scenario should be worked through (and standard design patterns applied to avoid it happening in the first instance)

Of course, in my analogy, once the PO noted said engines exploding it's up to the PO to stop using them and to demand a resolution, and not to just simply prosecute the drivers for breaking the engines. That remains inexcusable.

I do agree that PO should have been much more sceptical and much more questioning of Fujitsu, but my expectation is that many of the issues were downplayed by Fujitsu to avoid impacting bottom line - and given professional competencies and responsibilities I'd place by far the lion's share of the blame upon Fujitsu for so many bugs of such significance going into a live system.


* aside from ensuring that they bought the trucks from a reputable manufacturer who operated to required standards. Which was the case for PO with Fujitsu here., but which flags a point others have made - IT doesn't have that same degree of rigour as other disciplines such as Civil Engineering, where the required standards for Fujitsu and any product Fujitsu delivered would be more explicitly stated.

[Paddington Bear]

It not being lawful* for the government to account for Fujitsu’s poor performance on previous contracts when awarding new ones is the absolute cherry on the top. Peak Civil Service, if you don’t laugh you’d cry.

Appreciate these things aren’t directly comparable, but things like this are what really scares me about an AI dominated landscape.


*my understanding is the law changed last year so that they now can.

[inactionman]

It not being lawful* for the government to account for Fujitsu’s poor performance on previous contracts when awarding new ones is the absolute cherry on the top. Peak Civil Service, if you don’t laugh you’d cry.

Appreciate these things aren’t directly comparable, but things like this are what really scares me about an AI dominated landscape.


*my understanding is the law changed last year so that they now can.

It's about the only thing that's kept Capita in business.

[Sandstorm]

It not being lawful* for the government to account for Fujitsu’s poor performance on previous contracts when awarding new ones is the absolute cherry on the top. Peak Civil Service, if you don’t laugh you’d cry.

Appreciate these things aren’t directly comparable, but things like this are what really scares me about an AI dominated landscape.


*my understanding is the law changed last year so that they now can.

It's about the only thing that's kept Capita in business.

[Paddington Bear]

Another day, another clip of a PO investigator confirming that they didn’t write their witness statement

[Rhubarb & Custard]

From what I've seen of this, it just looks like bad coding by a large consultancy that put too many inexperienced (read: cheap) people into delivery and didn't support them properly.

As just one example of countless, the system would double-account transactions if it hung or otherwise crashed upon submission, without telling the user or allowing for correction. This is very unlikely to be a stated user requirement (to be clear, that it needs to be stated that the system needs avoid this situation), it should be a development standard to be able to manage nonrepudiation, message delivery and sequencing, and recovery of service outage, assuming that the underlying issue lies somewhere in that lot.

The user requirement would be to allow a sequence of debits and credits to be entered and for the system to process these, presumably in order of entry - the user would, never expect to say 'oh and don't fucking add the same transaction twice if the shoddy system falls over and forgets where it got to'.

Even despite this, the project team itself should elicit any technical requirements- e.g. recovery point objectives which say how far back the system needs to recover if it fails - the business user would have no clue whatsoever that this even needs to be thought about. The project team should perform all the unit and system tests to make sure it functions as cogent system, including performance and recovery - although this would be done by a dedicated test team, this team is generally a part of the project (security would be tested but the most stringent of that would be done for the most part by a third party, to avoid the project team marking too much of their own homework). The end user performs a User Acceptance Test that just warrant the system performs as required as part of the business process - they might do some negative tests e.g. try to log the same user on twice at the same time, deliberately mis-enter account numbers etc, and they may indeed pick up some of the issues that horizon suffered from, but these should be caught elsewhere. They certainly won't be expecting to test for technical fuck-ups of the type I've seen described.

In short, the main testing done by the end customer won't cover the situations that seem to have occurred which have caused the problems. Bog-standard good development practice should have prevented it, and it's not entirely unreasonable for the customer to expect the delivery partner to be actively managing this.

I appreciate the development team and testing team may have flagged many of these issues, but it seems they did not actually specify the criticality or the actual impact of these very clearly. I also appreciate this was all 20 years or more ago, but these are very basic practices.

Because, of course if they admit to shit development practices and intrinsic flaws then they ain;t getting paid.

I've suffered this far too many times in my professional life, hence my slightly shouty response. It's pretty much par for the course with many of these big consultancy and delivery practices - win the job, put the cheap grads on and blag/brazen the delivery, thus maximising profit and bonuses even if the customer does not get what they expected.


As a weak analogy, if I had an extension built for my house I'd check the room dimensions are as specified and do some basic checks that windows don't leak etc, but I'm not going to assess whether the wiring may lead to a fire risk over time or whether the foundations are likely to sag. The developer should be actively doing that, as it's their day job to know about it and the nuances of the build. Even if the developer reported technical issues to me, it's on them to tell me that 'this is a show-stopper' rather than let my house burn down or collapse in on itself. If the room sizes aren't right for my family, that would be my responsibility (or the architect's), not the developer's. That's where responsibilities lie.

Quite possibly.

But we still need to know what the communication operations processes were between the PO and their supplier before during and after the project, we need to know what was defined as the requirements, what the testing detailed, and what was signed off as going live.

And because the PO own the business I'm putting them first in the accountability stakes, at least for now, but I dislike the notion when you outsource work your outsource risk/responsibility. It's possible the PO could convince me they did everything that could reasonably be expected as due diligence on their part, but the impression they're giving doesn't support that. I'm much more inclined to think we had a bunch of people with a bad grasp of Classics, Theology and PPE running the implementation with little to no understanding of what they were doing, or at least asking for, and ignorance is a piss poor defence.

And of course the whole thing will be muddied further by the respective involvements of the Japanese and UK government.

I agree you don't/can't outsource all risk, but you certainly would expect the supplier to assume the responsibility of developing a coherent system that can manage very specific technical faults. A server losing contact with client (if that is the architecture) should not result in invisible transactions being added. It just shouldn't.

If the PO bought a fleet of trucks they'd assume responsibility that the trucks can take the loads safely and can be safely and effectively operated on UK roads, but they can't assume the risk of the engines exploding as the manufacturer got their metallurgy wrong*. In a number of cases it's that level of failure from Fujitsu, and would not be obvious test failures as some that I've seen documented would only occurred under a very specific set of conditions that couldn't be guaranteed to be be replicated under final acceptance test. They should have been picked up under unit test where that sort of scenario should be worked through (and standard design patterns applied to avoid it happening in the first instance)

Of course, in my analogy, once the PO noted said engines exploding it's up to the PO to stop using them and to demand a resolution, and not to just simply prosecute the drivers for breaking the engines. That remains inexcusable.

I do agree that PO should have been much more sceptical and much more questioning of Fujitsu, but my expectation is that many of the issues were downplayed by Fujitsu to avoid impacting bottom line - and given professional competencies and responsibilities I'd place by far the lion's share of the blame upon Fujitsu for so many bugs of such significance going into a live system.


* aside from ensuring that they bought the trucks from a reputable manufacturer who operated to required standards. Which was the case for PO with Fujitsu here., but which flags a point others have made - IT doesn't have that same degree of rigour as other disciplines such as Civil Engineering, where the required standards for Fujitsu and any product Fujitsu delivered would be more explicitly stated.

Again maybe.

But still the PO own the process of establishing and maintaining communication between the two, and we need to know what was said and when all along the process, and they own the process of testing (albeit you would think Fijitsu wold have found some of the reported bugs before handing it over to the client to find), and they own the process of signing the project off to go live. And it's simply not good enough to say the vendor convinced us it was okay/acceptible, they specifically own that process, they might not understand that, but again ignorance is no defence

[Dinsdale Piranha]

After several years of dealing with local and central government, It became clear that one of the core skills for surviving was the ability to slopey shoulder everything and find somebody to blame.

The Post Office are a government department. ICL were effectively a government department. Not taking responsiblity is ingrained in their DNA.

[Rhubarb & Custard]

After several years of dealing with local and central government, It became clear that one of the core skills for surviving was the ability to slopey shoulder everything and find somebody to blame.

The Post Office are a government department. ICL were effectively a government department. Not taking responsiblity is ingrained in their DNA.

Having worked in finance for a few decades it's not like you'd look around and think those responsible for a financial crash, fraud on a vast scale, massive misselling scandals pay much of a price. It's almost like those in charge of marking their own homework take a lenient view of their work whether public or private

[Dinsdale Piranha]

After several years of dealing with local and central government, It became clear that one of the core skills for surviving was the ability to slopey shoulder everything and find somebody to blame.

The Post Office are a government department. ICL were effectively a government department. Not taking responsiblity is ingrained in their DNA.

Having worked in finance for a few decades it's not like you'd look around and think those responsible for a financial crash, fraud on a vast scale, massive misselling scandals pay much of a price. It's almost like those in charge of marking their own homework take a lenient view of their work whether public or private

Finance is much clearer. Everybody's in it for the money. They're more likely to be evil rather than useless, incompetent layabouts. (based on ~20 years dealing with investment banks day to day)

[Rhubarb & Custard]

After several years of dealing with local and central government, It became clear that one of the core skills for surviving was the ability to slopey shoulder everything and find somebody to blame.

The Post Office are a government department. ICL were effectively a government department. Not taking responsiblity is ingrained in their DNA.

Having worked in finance for a few decades it's not like you'd look around and think those responsible for a financial crash, fraud on a vast scale, massive misselling scandals pay much of a price. It's almost like those in charge of marking their own homework take a lenient view of their work whether public or private

Finance is much clearer. Everybody's in it for the money. They're more likely to be evil rather than useless, incompetent layabouts. (based on ~20 years dealing with investment banks day to day)

In its defence it also has plenty of useless, incompetent layabouts. But when the senior useless incompetent layabouts get laid off at least they get utterly absurd severance packages

[Camroc2]

It not being lawful* for the government to account for Fujitsu’s poor performance on previous contracts when awarding new ones is the absolute cherry on the top. Peak Civil Service, if you don’t laugh you’d cry.

Appreciate these things aren’t directly comparable, but things like this are what really scares me about an AI dominated landscape.


*my understanding is the law changed last year so that they now can.

Fujitsu UK is the old ICL ( taken over in 2002), and would have had lots and lots of tentacles deep into all the UK public services for decades.

And apparently still do.

[fishfoodie]

It not being lawful* for the government to account for Fujitsu’s poor performance on previous contracts when awarding new ones is the absolute cherry on the top. Peak Civil Service, if you don’t laugh you’d cry.

Appreciate these things aren’t directly comparable, but things like this are what really scares me about an AI dominated landscape.


*my understanding is the law changed last year so that they now can.

Fujitsu UK is the old ICL ( taken over in 2002), and would have had lots and lots of tentacles deep into all the UK public services for decades.

And apparently still do.

The Register has a decent write up of how well they've done, even since the Horizon scandal broke, & as ever with the Reg, the comments are very illuminating too.

https://www.theregister.com/2024/01/11/ ... ocurement/

[Blackmac]

Odd hearing them talking about £600k compensation. Surely this is no more than a starting base as some have clearly suffered a lot more than others. Alan Bates himself has said he was one of the lucky one as he was able to maintain a reasonably decent lifestyle. Surely people who went to jail or whose partners committed suicide receive many times more.

[shaggy]

Odd hearing them talking about £600k compensation. Surely this is no more than a starting base as some have clearly suffered a lot more than others. Alan Bates himself has said he was one of the lucky one as he was able to maintain a reasonably decent lifestyle. Surely people who went to jail or whose partners committed suicide receive many times more.

Wonder how big a class action for this type of thing in the US would be? Many billions is my guess, not tens of millions as likely here.

[Blackmac]

Odd hearing them talking about £600k compensation. Surely this is no more than a starting base as some have clearly suffered a lot more than others. Alan Bates himself has said he was one of the lucky one as he was able to maintain a reasonably decent lifestyle. Surely people who went to jail or whose partners committed suicide receive many times more.

Wonder how big a class action for this type of thing in the US would be? Many billions is my guess, not tens of millions as likely here.

Yep, Fujitsu and the PO would be getting royally bummed by the US courts.

[SaintK]

This just gets worse and worse!

A former Post Office chair said he was told by a senior civil servant to stall compensation payouts to post office operators so that the government could “limp into” the general election.
Henry Staunton, who was sacked by the business secretary, Kemi Badenoch, last month amid anger over the Horizon scandal, said the request came soon after he took up the role in December 2022.
He also alleged that Nick Read, the Post Office chief executive, tried in January to dissuade the government from proceeding with blanket exonerations for operators.

https://www.theguardian.com/uk-news/2 ... n-payouts

[C69]

This just gets worse and worse!

A former Post Office chair said he was told by a senior civil servant to stall compensation payouts to post office operators so that the government could “limp into” the general election.
Henry Staunton, who was sacked by the business secretary, Kemi Badenoch, last month amid anger over the Horizon scandal, said the request came soon after he took up the role in December 2022.
He also alleged that Nick Read, the Post Office chief executive, tried in January to dissuade the government from proceeding with blanket exonerations for operators.

https://www.theguardian.com/uk-news/2 ... n-payouts

Hope this nameless Civil Servant is outed along with any Government member who suggested this.
À tsunami of filth is coming forward now from the Tories.

[fishfoodie]

This just gets worse and worse!

A former Post Office chair said he was told by a senior civil servant to stall compensation payouts to post office operators so that the government could “limp into” the general election.
Henry Staunton, who was sacked by the business secretary, Kemi Badenoch, last month amid anger over the Horizon scandal, said the request came soon after he took up the role in December 2022.
He also alleged that Nick Read, the Post Office chief executive, tried in January to dissuade the government from proceeding with blanket exonerations for operators.

https://www.theguardian.com/uk-news/2 ... n-payouts

Hope this nameless Civil Servant is outed along with any Government member who suggested this.
À tsunami of filth is coming forward now from the Tories.

Assuming this senior civil servant isn't currently off sick, & never intends coming back.

[C69]

This just gets worse and worse!

https://www.theguardian.com/uk-news/2 ... n-payouts

Hope this nameless Civil Servant is outed along with any Government member who suggested this.
À tsunami of filth is coming forward now from the Tories.

Assuming this senior civil servant isn't currently off sick, & never intends coming back.

Yes assuming that.

[Ymx]

This describes the nature of the bugs, and team

What was Horizon?

In layperson’s terms, Horizon was a till. More specifically, it was an electronic point-of-sale (Epos) system. It replaced the old paper-based tills that had been used in post offices across Britain with a new networked system. It was also the massive backend of those tills, that networked together the entire Post Office system.
What were the supposed advantages?

Replacing paper receipts with an electronic database should have saved post office operators time and effort, allowing them to manage their accounts at the push of a button. At the simplest end, for instance, the Horizon system could collate all the transactions over the course of a month and calculate how much cash was still expected to be in the Post Office’s coffers.
What went wrong?

The system was simply not up to the tasks placed on it. The Post Office knew as much as early as 1999, when trials run in preparation for the launch revealed “severe difficulties being experienced by subpostmasters”, the inquiry into the scandal heard.
One member of the development team, David McDonnell, who had worked on the Epos system side of the project, told the inquiry that “of eight [people] in the development team, two were very good, another two were mediocre but we could work with them, and then there were probably three or four who just weren’t up to it and weren’t capable of producing professional code”.
What sort of bugs resulted?

As early as 2001, McDonnell’s team had found “hundreds” of bugs. A full list has never been produced, but successive vindications of post office operators have revealed the sort of problems that arose. One, named the “Dalmellington Bug”, after the village in Scotland where a post office operator first fell prey to it, would see the screen freeze as the user was attempting to confirm receipt of cash. Each time the user pressed “enter” on the frozen screen, it would silently update the record. In Dalmellington, that bug created a £24,000 discrepancy, which the Post Office tried to hold the post office operator responsible for.
Another bug, called the Callendar Square bug – again named after the first branch found to have been affected by it – created duplicate transactions due to an error in the database underpinning the system: despite being clear duplicates, the post office operator was again held responsible for the errors.

[SaintK]

Deeper and deeper down the rabbit hole!!

David Cameron's government knew the Post Office had ditched a secret investigation that might have helped wrongly accused postmasters prove their innocence, the BBC can reveal.
The 2016 investigation trawled 17 years of records to find out how often, and why, cash accounts on the Horizon IT system had been tampered with remotely.
Ministers were told an investigation was happening.
But after postmasters began legal action, it was suddenly stopped.

It meant that over two years, the Post Office had spent millions of pounds on three separate reviews into remote access - Project Zebra, the Swift review and the 2016 Deloitte investigation - while publicly claiming it was impossible.
But all three were buried by the Post Office. Neither the Swift review nor Project Zebra were disclosed to sub-postmasters, depriving them of vital information that could have helped them in court; and the Deloitte investigation was halted before it could deliver its findings.

https://www.bbc.co.uk/news/business-68146054

[Margin__Walker]

In more unsurprising news, interaction between the Post Office's (independent and unbiased) expert Fujitsu witness and their barrister wasn't remotely above board in the trial that sent a pregnant post master to jail.

https://www.bbc.co.uk/news/uk-68606121

[shaggy]

In more unsurprising news, interaction between the Post Office's (independent and unbiased) expert Fujitsu witness and their barrister wasn't remotely above board in the trial that sent a pregnant post master to jail.

https://www.bbc.co.uk/news/uk-68606121

So clearly not an expert witness then.

[Enzedder]

Man, the Post Office lied to the courts - they knew that their defence was false.

Secret papers reveal Post Office knew its court defence was false

Imagine the hurt and anger coming up now if you were one of the 1000 postmasters affected. Public flogging and jail is too good for those buggers, and their lawyers

[petej]

Man, the Post Office lied to the courts - they knew that their defence was false.

Secret papers reveal Post Office knew its court defence was false

Imagine the hurt and anger coming up now if you were one of the 1000 postmasters affected. Public flogging and jail is too good for those buggers, and their lawyers

You need to nail senior management and some lawyers in this to change behaviour. To often the scapegoats are lower down the food chain.