For all that's holy, can you just leverage the Web, please?

When I moved in with my wife Laura in 2005, we lived in a shared apartment in Barcelona that had an ancient washing machine that was just there already, no idea who initially bought it. I managed to break the washing machine door's closing mechanism some time in 2006, so for a few weeks, whenever we did the washing, we had to lean a chair against the door so it wouldn't open. At the time, we were both students and living on a small budget.

Eventually, later in the same year, we bought an Electrolux machine that has accompanied us ever since. First on our move to Hamburg, then there through three apartments, and finally back to Spain, where we live now in the Catalonian countryside. Anyway, the washing machine had a motor damage last week, so after almost 20 years, it was time for a new one. I ordered it online (another Electrolux, without Internet nor WiFi), it was delivered swiftly, and I installed it hopefully correctly.

The washing machine came with a voluntary 10 year warranty if you registered it. The brochure where this offer was announced featured a free telephone number and a QR code that pointed at the number (in plain text, not making use of the tel: protocol). I called the number, and to my absolute surprise there were currently more callers than usual. After about 20 minutes, I had an agent on the phone, but after saying what I wanted, they just hung up on me (or the connection cut, whatever). Fine, I called again, but now, the call center was over capacity and they didn't even let me enter in the wait loop.

They did offer to send me a link to a chat service on their website via SMS, though, so I went for that option. The SMS literally pointed me at something like https://www. broken up by a space and then example.com/gc/. When I clicked the linkified example.com/gc/, I ended up on a broken site whose certificate wasn't trusted. After fixing the link manually and prepending the https://www. part, the page didn't load.

At this point I was close to giving up, but I had one last card that I wanted to play: I searched Google for "electrolux warranty register", and it pointed me at a site https://www.example.com/mypages/register-a-product/ as the first result. This looked promising. The mypages already suggested that this was gated behind a login, so I created an account, which was painless. (Turns out, after having an account and being logged in, the chat URL also worked—what an oversight on their part.) On the page, they had a field where you could enter the washing machine's product number from the identification plate on the door of the washing machine, together with helpful information where to find the data.

But even better, they offered a service where you could just upload a picture of the identification plate, and some AI on their server then extracted the product number and let you register the product with two clicks. What a fantastic experience compared to the crappy (and likely for the operator way more expensive) call center experience.

Why they didn't just put this URL on the brochure and the QR code is beyond me. As the title suggests: For all that's holy, can you just leverage the Web, please? Don't make me talk to people! They could still offer to register the machine by telephone as an alternative, but in 2025, the default for such things should just be the Web.

Bonus 🔗

Since I work on built-in AI as my day job in the Chrome team at Google, I could not not notice this "extract the product number from this identification plate" use case for client-side AI. I coded up a quick demo using the Prompt API embedded below that shows this in action. Here's a quick walkthrough of the code:

1. Create a session with the LanguageModel, informing the user of download progress if the model needs to be downloaded, and telling the model about the to-be-expected inputs (English texts and images) and outputs (English texts). In the system prompt, I tell the model what its overall task is (identify product numbers from photos of identification plates).
2. Prompt the model using the promptStreaming() method with a multimodal prompt, one textual and one image. The Prompt API supports structured output in the form of a JSON Schema or regular expression. Product numbers have nine digits, so I pass the regular expression /\d{9}/ as the responseConstraint option.
3. Iterate over the chunks of the response. Since I'm just expecting nine digits, this is probably a bit overkill, but, hey…
4. (Not shown) On the server, verify that the recognized product number actually exist. Companies typically have some sort of verification rules like checksums, or washing machine product numbers always start with 91 or something. If you know those rules, you can of course make them part of the responseConstraint, but you always need to verify untrusted user input (which the output of an LLM counts as) on the server.

const session = await LanguageModel.create({
  monitor(m) {
    m.addEventListener('downloadprogress', (e) => {
      console.log(`Downloaded ${e.loaded * 100}%.`);
    });
  },
  expectedInputs: [{ type: 'text', languages: ['en'] }, { type: 'image' }],
  expectedOutputs: [{ type: 'text', languages: ['en'] }],
  initialPrompts: [
    {
      role: 'system',
      content:
        'Your task is to identify product numbers from photos of identification plates.',
    },
  ],
});

const stream = session.promptStreaming(
  [
    {
      role: 'user',
      content: [
        {
          type: 'text',
          value:
            'Extract the product number from this identification plate. It has nine digits and appears after the text "Prod.No.".',
        },
        { type: 'image', value: image },
      ],
    },
  ],
  {
    responseConstraint: /\d{9}/,
  }
);

for await (const chunk of stream) {
  console.log(chunk);
}