Change control — Friday 14 August

What it is Someone in the building builds the cart for a customer, sets the shipping price, and gets a link to send. The customer opens the link, sees the parts and the total, and pays by card or a wallet like Apple Pay. It covers the three orders that fall out of the normal checkout into a phone call today: an address outside the US, a part that needs freight, and an order taken over the phone. The order then arrives the same way a website order does, and prices are checked again at the moment the customer clicks. The public checkout is untouched.

How we test it Sign in as staff, build a cart on the staff quotes page, send the link, pay it with a test card, then confirm the order appears with a number and the confirmation email arrives. That one real order placed end to end is what closes the work; 948 payment tests pass meanwhile, 36 of them new.

Evidence Asked straight at both payment services, so the store and its cache are out of the path entirely.

No picture, and it is not usable yet. Nothing answering, then an address that answers with a real message, is a genuine before and after, and the control rules out a catch-all. It stops there. Quotes are switched off on the test copy, so no link could be produced, no payment made and no order looked for; the staff form needs a staff sign-in we cannot drive, which is why there is nothing to photograph. A link cannot be paid at all until Alex adds the table that holds the quotes. There will be a picture and a real order here when it is switched on.

Roll back or fail forward Roll back — new pages only, nothing a shopper uses today depends on them, and no link has gone out from the store yet.

What it is When a shopper asks what shipping costs, the answer is now sent the moment it is worked out. Before, we held the answer while also saving a copy of it, and on a server whose database connection had gone bad that save took twenty to thirty seconds, so the shopper waited that long for a number that had been ready in one second. That twenty-to-thirty-second reading is the one recorded on 11 August by the person who made the change, not ours, and it could not be reproduced today because it needs a server in that state. The copy is still saved, it just no longer keeps anyone waiting, and a failed save now raises an alarm instead of passing silently.

How we test it 235 delivery tests pass, and the two new ones fail on the old code for the right reason. With the database unreachable the quote still comes back at once.

Evidence A real shipping question — Clinton NY to Mountain View CA, one five-pound package — asked three times of each service.

There is no before and after here today, and the numbers above must not be read as one. The twenty-to-thirty-second figure came from one particular server whose database connection had died; that server fault is a separate piece of work and this change does not fix it. A healthy server already answers in about a second, which is what was measured on both sides. What this change does is stop the slow save from ever reaching the customer, and that only shows itself when the save hangs — a condition we cannot create on demand. Read prod's 1.2 seconds as "prod is healthy right now", not as the before.

Roll back or fail forward Roll back — one small change on one path, and the old behaviour was slow rather than wrong.

What it is John ruled on Monday that the only thing that says a part is in stock is a real count at a CNH depot. Until now a part could show as in stock and buyable because CNH listed it as something we could order, even with zero on the shelf at every depot, and 127,404 parts were in that state. Now the count decides. Only 1,052 parts actually lost an Add to Cart button, because the rest were quote-only pages that never had one. Of the 302,665 parts that stay in stock, 11,972 sit on Clinton's own shelf and ship same day, and 290,693 come from a CNH depot with handling time.

How we test it Open SBA185186320, John's own example. It reads out of stock and has no Add to Cart button; his screenshot shows all four depots at zero.

Evidence The part page and the backing record read on both sides on Wednesday.

Identical. Both stores already show this part out of stock with no Add to Cart, which is the behaviour the change is about.

Zero difference on every surface, so nothing was measured. Both sides already read out of stock with no Add to Cart, which is the after. This change went to the store on Wednesday, ahead of this release, so the before it describes is not observable anywhere today. This particular part is also a poor witness: with the depot count at zero the old rule and the new rule agree, so even a live before-and-after window would show no difference on it. A part where the two disagree would be needed, and none is named.

Roll back or fail forward Fail forward — a rollback re-advertises stock we do not have, and this is John's own ruling.

What it is Two smaller places still answered the old question. The in-stock count in the filter list beside search results was built from a different source than the badge on the part itself: searching seal, it claimed 183 in stock while the page returned 18,231 results. It now counts the same way the badge does. The small in-stock tag in the type-ahead dropdown was removed rather than corrected, because the lookup behind that dropdown never receives the depot count and cannot answer honestly.

How we test it Search for seal. The in-stock number in the left filter list reads about eighteen thousand, the same size as the in-stock half of the results, instead of the 183 it used to claim.

Evidence The same search run on both stores and both search services on Wednesday.

Identical, because this one is already on the store. The in-stock number in the left filter list is the thing to look at.

No before-and-after window. The store already serves the corrected number, because this went out on Wednesday. The gap between 18,149 today and the 18,163 recorded when the work was done is two days of catalogue movement, not a difference between the two copies. One thing this change does not close, and does not claim to: the in-stock and out-of-stock counts add up to 30,083 against a total of 31,947, leaving 1,864 results in neither bucket. That gap is the same on both sides and is a separate question.

Roll back or fail forward Fail forward — a rollback restores counts wrong by 127,000 sitting next to the corrected badge.

What it is Alongside every part we handed out two extra values that no longer agreed with the page. The first was a ready-made instruction naming which button to show; it was built only from whether the part could be quoted, with no stock input at all, so it said Add to Cart on 835 parts that the very same answer badged out of stock. The second was a stock-like value copied straight from CNH's own list, which disagreed with the badge on 127,490 parts. Neither was being read by anything — checked across the store, the order services and the CNH service — because the store and the Google feed work the button out for themselves. Both are deleted rather than repaired.

This is invisible to a shopper and to Google, and that is the point: the badge, the price and the buy button are untouched. What it removes is two ways for a future page to pick up the wrong answer.

How we test it Open SBA185186320, 83924760 and SBA340293195 before and after. The stock line, the price and the button are identical on each; only the unused values are gone.

Evidence None beyond the counts above and the three parts named. There is no visible surface to photograph, which is itself the claim being made.

Roll back or fail forward Roll back — a deletion of three unused values in total, and putting them back is instant and harmless.

What it is A New Holland part we price by request was showing one cent, and the same one-cent price was being sent to Google Shopping. Its neighbour with the same status but no price behaved correctly. Now the status itself blocks the sale, the placeholder price, the price Google reads on the page and the Google Shopping feed, so both parts read Contact for price. Denis found this one and wrote the failing test that came with the fix.

How we test it Open the New Holland part CT-NHL-06404120C: it must read Contact for price with no buy button, and its neighbour CT-AGC-000050147 must be unchanged.

Evidence Both parts read on both stores, with the cache bypassed, plus what Google is given on each page and the backing record.

The one-cent price and the offer Google could index are gone, with a same-status control that did not move — that pairing is what makes this evidence rather than an assertion. The 83 extra pixels on a phone are explained and need no further story: the parts-desk sentence and the wider Ask for a price button are taller than "$0.01" plus "Notify me when back". The desktop height is unchanged.

One half is unproven today. This part is out of stock on the store right now, so the store already shows "Notify me when back" rather than a working buy button. The claim that a buy button was removed does not reproduce on 13 August; what can be evidenced is the price and the offer Google reads.

Roll back or fail forward Fail forward — rolling back puts a one-cent price and a buy button back on parts we will not sell.

What it is On a desktop the Brand filter on the parts search page offered a longer list than the page itself could account for: fifteen brands to tick against nine the search actually reports. The desktop list now matches the search, and matches what a phone has always shown.

How we test it Open the parts page on a desktop and count the brands in the Brand filter, including the ones behind "Show more": fifteen before, nine after, and the nine are exactly the brands the search reports.

Evidence Both stores read with the cache bypassed, counting the tick-boxes and the "Show more" residue, then checking that total against the brand list the page itself carries.

Two claims made for this change do not survive measurement, and are not made here. The work was described as removing four brands that always returned nothing. Only one of the four, Ferris, is actually gone; Ventrac, McHale and Marcrest are all still in the filter, each with thousands of parts behind them. And we could not measure what ticking a brand returns at all: the tick-box is not part of the web address, so it cannot be asked for from outside. The address that looks like it should stand in for a tick does not agree with the page's own numbers, so we are not quoting it. What is measured, and reconciles, is the list going from fifteen to nine.

Roll back or fail forward Roll back — one screen area, no data change; the longer list comes back.

What it is Two opening animations were being applied to the header search panel at once, and which one you saw came down to the order the styling happened to compile in. Today it looked right by luck. Only one is applied now, so it cannot change on us the next time something unrelated is rebuilt. Nothing looks different on screen.

How we test it Open the header search on a desktop and watch the panel appear: the same motion as today, with only one animation on the panel.

Evidence None was taken, and none would show anything — the whole claim is that the panel looks and moves exactly as it does now. What changes is that it will keep doing so.

Roll back or fail forward Roll back — a styling change on one panel.

What it is A part number can belong to more than one maker. When a Kuhn or Ferris part happened to share its number with a New Holland part, our catalogue quietly copied the New Holland price and shelf count onto it. That is now stopped: a part shows its own maker's price, or none at all. On the live catalogue that removed a borrowed price from 5,927 parts and a borrowed shelf count from 37,077, and left every New Holland part exactly as it was.

How we test it Open the part page for 504189487 (Ford). No price is shown and it reads out of stock. New Holland part 47927519 still shows $91.36.

Evidence The part page read on both stores, and — because the backing record is the truth here — the record itself read from both search services.

Identical, because this one is already on the store. The pair is here to show the price and the stock line as they now read.

No before-and-after window exists for this one, on either surface. The two sides answer identically, which means the store is already serving the corrected behaviour and the before is gone. That fits how this change works: it corrects how the catalogue database joins two tables, and both services read the same database, so applying it moved both at once. We could not confirm exactly when the store's search was last rebuilt — the tool that reports it timed out twice — so this rests on the two services answering identically, which is the stronger evidence anyway. The price shown above is not itself proof either way; there is no pre-fix reading to compare it against.

Roll back or fail forward Fail forward — a rollback puts other makers' parts back on New Holland prices, and it is already live and quiet.

What it is The diagram tab opens the largest equipment type by itself. When that type came back with no models, as New Holland Construction does, the shopper got the full page frame with every count reading zero, an empty grid, no message and nowhere to go. It now falls back to the category picker with real counts, so there is always something to click. A shopper who asked for that type by address still gets its honest empty answer.

How we test it Open the New Holland Construction diagrams page while its model list comes back empty: it must show clickable categories with counts, not an empty grid reading zero.

Evidence Both stores read with the cache bypassed, comparing the visible text, the category tiles and the counts beside them.

Identical, down to the pixel. The empty state this change fixes is not happening on either store today, so what the pair shows is that the working state is untouched.

This one could not be measured, and the honest result is no difference at all. The broken state this change fixes is not happening today: the model list comes back with 404 models on both sides, so the store is already showing clickable categories with counts. That is a could-not-measure, not a proof of no change — the broken path was never entered on either side, so nothing about the fix was tested and nothing about it was disproven.

Roll back or fail forward Roll back — one screen area; the dead empty grid returns.

What it is Two tiles on the equipment page, Skid Steers and Construction Equipment, pointed at an agriculture address that immediately redirected to the construction section. They now link straight there. The same redirecting link also sat in the browse block on the parts page, where Google was following it on every crawl; it is gone. The construction section is still reachable from the brand links beside it.

How we test it On the parts page count the agriculture browse tiles: eleven, with no construction tile. On the equipment page the two construction tiles open the construction section with no redirect.

Evidence Both pages read on both stores with the cache bypassed, counting the distinct agriculture browse links.

The two pages look the same, and that is the point: what changed is where a tile points, not what it shows. The pair is here as proof that nothing else moved.

Twelve to eleven with construction the one removed matches what was intended, and it cross-checks against a completely separate count taken for the browse-block change further down this page, on the same two documents.

The open half of this is now closed. Reading the equipment page in a browser on both stores, the store carries two construction links — the construction section, and one routed through the agriculture address — while the test copy carries only the first. On its own a single link's difference on a page that has a week of other changes behind it would be weak; it is the agreement with the independent count that makes it evidence.

Roll back or fail forward Roll back — link targets only; the redirect the site already serves stays as a safety net.

What it is Part pages for Ferris, Kuhn, McHale and Marcrest are deliberately held back from Google while their data matures, and each page says so itself. They were still listed in the address lists we submit to Google, from two separate files, which is exactly the contradiction Search Console reports as a submitted page marked not-to-be-indexed. Both files now leave them out. Nothing changes for a shopper.

How we test it Count held-back brand pages in the parts address list and the image address list: 358 before, none after. Then watch the submitted-but-not-indexed count in Search Console fall.

Evidence Both address lists read on both stores, counting entries per page rather than raw text matches.

The before is exact and checks out against itself: 123 plus 163 plus 41 plus 31 is 358, and the identical 358 appears in both files, which is precisely the "from two places" this change is about.

The after cannot be observed anywhere before Friday. The test copy serves no address lists at all, because it is deliberately closed to Google. That is a missing file, not a count of zero, and it must not be reported as zero. The only place this change can be confirmed is the store itself, after the promote.

Roll back or fail forward Roll back — a filter on two files, no data change.

What it is The browse-by-type block on the parts and brand pages arrived after the page did. With scripts off you saw eighteen grey placeholder tiles that never filled, and on the parts page nothing at all. That block is the path Google follows to every brand-and-type page, so none of those links were in the first delivery. They are in the page itself now: eleven labelled links with model counts, no placeholders, and one fewer deferred section on each page.

How we test it Fetch the brand page as a crawler and count browse links in the first delivery: 0 before, 11 after, with no grey placeholders. Load it with scripts off and the tiles are real links.

Evidence Each page was split at the point where the first delivery ends and the later-filled part begins, and the links counted on each side of that line.

The brand page, the second surface this change touches. Once loading has finished the two look the same; what moved is that the browse links now arrive with the page instead of being filled in afterwards.

This is the cleanest measurement of the set and it agrees with itself three ways: the links move from the later half to the first delivery, the number of places waiting for later content drops by exactly one on each page, and the total link count is preserved apart from the one link deliberately removed. It is worth saying why the obvious check is a trap — searching the whole page finds the links on the store too; they are simply in the half that arrives afterwards.

What this does not prove. We measured what the two stores send, not what a browser with scripts switched off actually draws, so the eighteen grey placeholders going to none is unverified here. The pair of pictures above is identical, which is expected and is not a null result: they were taken with scripts on, and once the scripts have run the block occupies the same space either way. The change is in what arrives first, which a picture cannot show.

Roll back or fail forward Roll back — one page area, and note it puts the type lookup on the critical path if search hangs.

What it is One setting on the account area hit a known fault in the site framework: it switched off preparing pages in advance for everything under Account. Links to order history, addresses and the rest each waited on a fresh request instead. The setting is removed. The signed-in measurement of how much quicker it feels is still owed, so we are describing the mechanism, not a number.

How we test it Signed in, click through Account, Orders and Addresses with the network panel open: the next page should already be requested before the click lands.

Evidence What could be checked from outside is the safety half — that a signed-out visitor is still turned away before anything renders.

This confirms the property that actually matters for safety: removing the setting did not open the account area to anonymous visitors, and the guard still fires before the page is built, on both sides.

It proves nothing about the benefit. Preparing pages in advance inside the account is a signed-in behaviour that happens in the browser. From outside, with no sign-in, both copies look the same and there is no observable before and after.

Roll back or fail forward Roll back — one line; the account area stops preparing pages in advance again.

What it is When someone signed in who had never saved a cart, the cart service answered that there is no cart, and the site treated that as the service being down. Nothing broke for the shopper, but every one of those first visits was recorded as a live error, which buries the real ones. It is now read as what it is, an empty cart.

How we test it Sign in with an account that has never saved a cart and load any page: the cart is empty and no cart-service error is recorded.

Evidence None was taken. This needs a signed-in account that has never saved a cart, which is a state we can create but not observe from outside, and the thing being removed is an entry in our own error log rather than anything on screen. A shopper sees the same empty cart before and after.

Roll back or fail forward Roll back — one small change to how one answer is read.

What it is When we build the address lists we submit to Google, we ask our own search service for the parts in batches. We were asking for ten at once against a service that accepts two, so four to eight of every ten were turned away and retried, roughly four hundred a day. It now asks for two at a time. The lists themselves come out identical.

How we test it Build one address list and count the rejections on the search service in that window: zero, rather than four to eight per ten batches.

Evidence None was taken on the test copy, for the same reason as the change above it: the test copy is closed to Google and serves no address lists at all, so there is nothing there to build. The before is the rejection count on the live search service; the after can only be counted after Friday's promote. What a shopper or Google sees does not change either way — the lists come out the same, and their fail-safe is untouched.

Roll back or fail forward Roll back — one number; the daily self-inflicted rejections return.

What it is On a good day a shopper sees nothing here. One shipping-quote server had lost its database connection and stayed in service for thirteen and a half hours, slowing quotes for everyone who reached it. Nothing caught it, because the short health answer it gave always said fine no matter what. Five changes give the same treatment to five servers, and the differences between them are deliberate.

The shipping-quote server now says it is unwell when its database is unreachable and restarts itself after about two minutes of that. The payment server, which sits on the same checkout path, stops receiving traffic after about thirty seconds and restarts after about three minutes — deliberately slower, because a payment can be in flight and a restart would drop it. The parts catalogue server and the accounts server get the same, and the build now refuses to ship any server that is missing its checks or has one set up wrongly, with three servers deliberately left out and the reason for each written into the check rather than left implied. The image server used to answer ready no matter what; it now genuinely reaches the image store before answering, within a three-second cap. The catalogue-book servers had the opposite problem — they took themselves out of service entirely whenever the image store hiccuped, even though everything they serve from the database was fine; storage moved to a separate deeper check, and that deeper check is also what a new version now has to pass before it takes traffic, replacing a release gate that checked nothing at all.

How we test it Describing each live server shows the restart check on it. Shipping quotes: 221 tests pass, with the same two pre-existing network failures as before. Payment: 861 pass, 6 fail — the same six local-environment failures as before. The build guard reports five servers required and twelve checks verified, and removing one server's checks makes it fail by name. Images: 13 tests pass against 8 before, and the new ones fail on the old code with exactly the wrong answer it used to give. Catalogue books: with storage down the ordinary check stays green and the deeper one goes red, pinned by tests.

Evidence

The store has no health checks at all today, on any server. That is the before, and it is the reason this group exists. The check that stops traffic reaching a bad server has been proven on one test-copy server only — the image one. Everything above lands on the store for the first time on Friday.

Roll back or fail forward Roll back — settings plus a few health answers and a build check; every server runs exactly the same without them, which is precisely the state we are leaving.

What it is Four servers were slowly eating memory until the platform killed them, and the accounts server was killed five times in fourteen days, most recently on 11 August. The cause was a connection setting that threw away every idle database connection each minute and immediately rebuilt it, roughly forty-seven thousand rebuilds a day against a handful of real requests. Putting that setting back to the maker's own default cuts the growth by roughly forty to seventy per cent. It is not the whole story: the rest is a known bug in the runtime we use, fixed in a version we have not moved to yet, so restarting a server stays the fallback while it is needed far less often.

How we test it Two live servers run side by side on the same size, booted sixteen minutes apart: the new setting climbed 1.91 points an hour, the old one 4.69. A build check fails if the old setting comes back anywhere.

Evidence The side-by-side above is the measurement, and it was taken on live servers rather than on the change itself. It is honest about its own limit: it halves a climb rather than ending it, and the cure is a runtime version we have not yet moved to.

Roll back or fail forward Roll back — a connection setting returned to its maker's default; one value either way, no queries and no pages change.

What it is The Kuhn dealer portal has never answered for 48,709 of the Kuhn parts in our catalogue, and it is not a gap we can close: only 96 of them appear in the dealer stock file and none carry replacement numbers. They are now marked as parts we cannot sell, so the nightly stock check stops re-asking about them, which had been costing 25,000 calls a night. Shoppers see no change yet. Whether these should also drop off the catalogue pages is still an open question.

How we test it Counted directly on the live database: 54,959 of 236,680 Kuhn parts marked, matching the saved list of 48,709 newly added.

Evidence The count above was read from the live database, which is also where this change was applied — it did not travel through a release, so there is nothing for Friday to promote. It is listed here because it belongs to this week's work and because it changes what the nightly check does.

Roll back or fail forward Roll back — the 48,709 part numbers were saved to a list, so reversing it is exact.

Four of these seven — the first four rows — are already on the store, moved there on Wednesday 12 August ahead of this release. The remaining three go out on Friday.