32 posts with tag “Apple”

LLMs make good analysts, bad oracles

I’m not an AI apologist by any means, but I’m frustrated by the muddled way LLMs have been marketed, portrayed, and used. I want to focus on the utility of them here, rather than the moral or legal implications of using copyrighted content to feed their corpora.

One of the first things we started doing when ChatGPT became public was, naturally, asking it questions. And for the most part, it gave us some pretty good answers.

But as has been widely demonstrated recently — as Google, Meta, and others have begun grafting “AI” onto their search results — is just how wrong it can get things, and it has had us asking: Is this really ready for widespread adoption? Asking for arbitrary bits of information from an LLM’s entire corpus of text — like Google’s and Bing’s smart summaries — is demonstrably, hilariously, and sometimes dangerously flawed.

Over the last couple years, I haven’t really heard much from OpenAI themselves about what we are supposed to be using ChatGPT for. They seem more interested in creating the technology — which no one could seriously doubt is impressive — than in finding real-world applications for it. The announcement for ChatGPT didn’t tell us what to do with it (though it did emphasize that the tool can be expected to product false information).

I think the misconception about what ChatGPT is purported to be good at can be attributed to the name and the UI. A chat-based interface to something called “ChatGPT” implies that the product is something it isn’t. It’s technically impressive, of course, and makes for a good demo. But chat doesn’t play to its strengths.

The reason any given LLM is even able to wager a guess at a general knowledge question is the corpus of text it’s been trained on. But producing answers to general knowledge questions is a side-effect of this training, not its purpose. It isn’t being fed an “encyclopedia module” that it classifies as facts about the world, followed by a “cookbook module” that it classifies as ways to prepare food. It was designed to produce believable language, not accurate language.

Where it does excel, however, is at coming to conclusions about narrow inputs. Things like Amazon’s review summaries; YouTube’s new grouping of comments by “topic”; or WordPress’s AI Feedback — these take specific streams of text and are tasked with returning feedback or summaries about them, and seem to work pretty well and have real utility.

These examples demonstrate two similar but distinct facets of LLMs: Their role as general knowledge engines, or “oracles,” and as input processing engines, or “analysts.” When we ask ChatGPT (or Google, or Meta) how many rocks we should eat per day, we are expecting it to behave as an oracle. When we ask it to summarize the plot of a short story or give us advice for improving our resume, we are expecting it behave as an analyst.

Signs point to Apple using LLMs primarily as analysts in the features to be announced at today’s WWDC, processing finite chunks of data into something else, rather than snatching arbitrary knowledge out of the LLM ether.

The allure of ChatGPT as an oracle is of course hard to resist. But I think if we recognize these two functions as separate, and focus on LLMs capabilities as analysts, we can wring some value out of them. (Their environmental impact notwithstanding.)

One Response

Apple TV+’s takeover of the Apple TV app Home Screen

Part of what used to be great about the Apple TV is that — unlike an Amazon Fire TV, for instance — Apple didn’t really have specific content it was trying to push on you. Sure, it might suggest that you rent or buy a movie from iTunes instead of from Amazon; and it would be safe to assume it might only recommend content that was available on iTunes, but practically that included everything.

Amazon, on the other hand, since they had gotten into the original content market, would be incentivized to give priority to their TV shows over others. Is my Amazon Fire TV recommending Sneaky Pete to me because it’s similar to stuff I watch, or because they stand to gain from it?

When Apple TV+ was announced, I worried that Apple’s having a stake in what specific content I watch would taint the Apple TV experience.

Boy, has it.

Below is a full view of the contents of the macOS TV app. The TV app on the Apple TV streaming box is roughly identical.

Continue reading

Leave a Comment

Apple’s incomplete pronoun fields

In iOS 17, you are now able to add pronouns to contacts, including your own contact card. This is a good thing, but at least one important pronoun case is missing from the “English” options.

The cases included are:

  1. Subjective (“Yesterday, she went outside.”)
  2. Objective (“I went with her.”)
  3. Possessive pronoun (“The idea was hers.”)

These three cases mirror the common “she/her/hers” structure used to communicate pronouns on social platforms like Twitter and Zoom; as such, they may seem complete, but they aren’t.

The missing case is the possessive adjective: “It was her idea.”

This may seem redundant, because in the declension of feminine pronouns, the possessive adjective is the same as the objective: “her.”

But in masculine pronouns, the possessive adjective is the same as the possessive pronoun: “his” (as in, “It was his idea”).

CaseFeminineMasculine
SubjectiveSheHe
ObjectiveHerHim
Possessive pronounHersHis
Possessive adjectiveHerHis

This isn’t a problem when interacting with people — we know how to decline the common feminine and masculine pronouns, so we know which form to use for the possessive adjective.

But because the iOS 17 UI doesn’t have a field for the possessive adjective, the OS and apps that have access to these fields — which have to behave programmatically — can’t know what to use for that case.

If I’m writing an app and I want the UI to say, “Would you like to call her?” in reference to some contact of yours, I can use the objective pronoun field. But if I want the UI to say “Her birthday is coming up” (the possessive adjective), I don’t have the necessary information.

I can try to guess, based on what I know about the declension of these common pronouns. But what about uncommon ones, like Ze or Xe? Anything I want to use will have to be hard-coded.

This is not even to mention the reflexive case — “She was proud of herself” — which is entirely absent.

Leave a Comment

Third-party gyroscopic head tracking is not “Spatial Audio”: the Anker soundcore Liberty 4

From a credulous 9to5Mac review of some new Anker earbuds:

Arriving with active noise cancellation, Anker also backs its Liberty 4 with Spatial Audio support, thanks to the internal gyroscope that helps immerse you in the sound.

From the description on the Amazon product page, titled “soundcore by Anker Liberty 4, Noise Cancelling Earbuds, True Wireless Earbuds with ACAA 3.0, Dual Dynamic Drivers for Hi-Res Premium Sound, Spatial Audio with Dual Modes, All-New Heart Rate Sensor”:

360° Immersive Spatial Audio: As you listen to music and watch movies, the built-in gyroscope and spatial audio algorithm track your head movements to always keep you at the heart of the sound for a completely immersive experience.

From the FAQ on Anker’s product website:

How do I enable Spatial Audio?

  1. Wear the earbuds.
  2. Enter the soundcore app and connect Liberty 4.
  3. Select Spatial Audio. 
  4. Spatial Audio cannot be turned on while sound effects are turned on. 
  5. Select Movie or Music mode to adjust the sense of distance while listening to different types of audio content.

Spatial Audio — the Apple feature — is an OS-level capability for taking true many-channel audio and “shaping” it to mimic the effect that the contours of your ears have on physical sound waves when they reach your head. Thus a sound in a movie that would be directed to the rear right speaker in a theater or surround sound system will sound like it is coming from behind and to the right of you when you are wearing compatible headphone hardware.

In other words, if you wear the Anker soundcore Liberty 4 earbuds, connect them to your iPhone, iPad, Mac, or Apple TV, and watch a movie that contains Dolby Atmos audio — you will not be hearing that movie in Spatial Audio. You will be hearing it in stereo, with the Liberty earbuds (on platforms where the app is available) shaping the waveforms so that they sound like they are originating from a space in front of you rather than the center of your head. No sounds will appear to come from behind you with this technology, as they can with true Spatial Audio.

Dynamic Head Tracking — the Apple feature — is a joining of hardware and software, whereby the source of your audio (your iPhone or iPad) is “aware” of its spatial relation to your headphones. Thus if you turn your head, the audio will continue to sound as though it’s coming from the device, rather than in the direction you’re facing; similarly, if you move the device around your head, the perceived locus of the audio will follow the device.

If you wear the Anker soundcore Liberty 4 earbuds and listen to music on your iPhone or iPad, even with the soundcore app, the experience is less clear to me. I believe that the app may ask permission to read your device’s motion sensors, giving it enough data to cause the audio to always follow the device, but I can’t be sure. In any case, this will not be the system-level Dynamic Head Tracking that is only possible with iOS 15+ and compatible Apple headphones.

It bears repeating that Spatial Audio and Dynamic Head Tracking are two separate things, but now it seems necessary to add that there is no Spatial Audio without Apple hardware, period.

There’s no technical reason that Apple couldn’t make this kind of third-party Spatial Audio compatibility possible, as far as I can tell, but as of yet, they haven’t done so.

Leave a Comment

Oh, Gee

Yesterday I became aware of the awkwardly named “The OG App” (🙄), which promises “the OG Instagram experience”: “With the OG app, you can easily filter what you see in your Instagram feed, create custom feeds, remove reels, ads, suggested content, and more!” It seemed impossible, and that’s because it is.

I nearly gave the app access to my Instagram account, believing for a moment that maybe, given its presence in the App Store at all, it had been properly vetted; but because of their suspiciously slow homepage, as well as the company’s bizarre name (“Un1feed”) and the dearth of info about it, I decided to wait. I’m glad I did.

Continue reading

Leave a Comment

iPadOS 16 Will Not Include Arbitrary Windowing

A discovery in some public WebKit code referencing MULTITASKING_MODE has some people breathlessly anticipating what they consider the obvious and long-overdue solution to multitasking on the iPad: arbitrary windowing:

And as it stands, the Magic Keyboard is more of a convenient desktop accessory than a productivity tool, but giving it a new interface would make it far more useful. A desktop or pro mode would instantly change that.

Google does something similar with its Chrome tablets, but Apple could do it better with a hybrid macOS-iPadOS environment that seamlessly switches between tablet and desktop mode while unlocking the benefits of a touchpad with an intuitive, powerful interface.

Macworld

We’ve all been burned many times by hoping a new iPadOS update will bring significant improvements for power users, but this does seem to be pretty solid evidence suggesting iPadOS 16 could be the year this finally changes.

9to5Mac

More recently, Mark Gurman wrote for Bloomberg:

The iPad’s next major software update, iPadOS 16, will have a redesigned multitasking interface that makes it easier to see what apps are open and switch between tasks, said the people, who asked not to be identified because the changes aren’t yet public. It also will let users resize app windows and offer new ways for users to handle multiple apps at once.

Admittedly, multitasking on the iPad has always been far from perfect. iOS 15 went a long way, I think, to solving some of its issues, but it’s still clunky and confusing.

But the solution is not and will not be “arbitrary windowing”: the ability to resize apps to any size and layer them on top of each other indefinitely.

Continue reading

Leave a Comment