{"slug": "meta-s-ships-facial-recognition-on-smart-glasses", "title": "Meta's ships facial recognition on smart glasses", "summary": "A security researcher discovered that Meta's smart glasses companion app, Stella, contains a fully functional on-device facial recognition pipeline, including three face models, a local database, and a biometric vector index. The researcher demonstrated the pipeline works end-to-end by triggering a \"Person Recognized\" notification on a test image, though the feature remains dormant for ordinary users with no visible UI or server-pushed identity data. The finding reveals Meta has assembled the complete technical infrastructure for real-time facial identification through its smart glasses, gated behind a server-side switch.", "body_md": "# Meta's smart glasses companion app ships a complete, dormant face-recognition pipeline on a stock account.\n\nStella is the companion app for Meta's smart glasses. Inspecting version `273.0.0.21`\n\nof the Android build (`com.facebook.stella`\n\n), I found the entire computational and storage stack for on-device facial recognition: three face models, a local database schema, a cosine-similarity vector index dimensioned to match the models, a write path that stages biometric records to disk, a fully wired notification surface, and a user-facing \"Connections\" widget.\n\nI want to be precise about what that does and does not mean, because the gap between the two is important.\n\n**What I can demonstrate:** the machinery is present, it is wired together. Several facial extraction and facial fingerprinting models are present and I was able run the recognition pipeline end-to-end on a test image and it detected a face, generate a 2048-dimension biometric embedding, searched a local index, and on a match fired an Android notification stating to the user \"Person Recognized\".\n\nTo get the pipeline to run I invoked its existing handler directly with a test photo.\n\n**What I cannot demonstrate:** that any of this is active for ordinary users. On a stock, unenrolled account the user-facing UI does not appear, and the screen the recognition notification deep-links to is missing from the build. I also did not observe Meta server-pushing identity data to the relevant database on my test account.\n\nSo this is not \"Meta is secretly identifying the people you look at.\" It is: the complete apparatus to do exactly that is sitting on the device, assembled and functional, gated by Meta.\n\nAll findings below are reproducible against `com.facebook.stella`\n\nv273.0.0.21.\n\n## Three face-recognition models ship on the device (~100 MB)\n\nThree ExecuTorch (`.pte`\n\n) models arrive on the device via NMLML, Meta's asset-delivery system, downloaded from Meta.\n\n| Asset name (Meta's naming) | File | Size | Function |\n|---|---|---|---|\n`android_facerec_scrfd` |\n`SCRFD.pte` |\n3.4 MB | Detects faces in an image |\n`android_facerec_kps_aligner` |\n`KPSAligner.pte` |\n117 KB | Crops and aligns each detected face |\n`android_facerec_sface` |\n`SFace.pte` |\n96 MB | Converts a face into a 2048-number embedding (the biometric fingerprint) |\n\nThese map onto open-source architectures, the same model families that other apps and academic projects already use:\n\n**SCRFD*** Sample and Computation Redistribution for Efficient Face Detection*(InsightFace, ICLR 2022). Reference implementation:`github.com/deepinsight/insightface`\n\n.**SFace*** Sigmoid-Constrained Hypersphere Loss for Face Recognition*(Zhong et al., 2021). Reference:`github.com/zhongyy/SFace`\n\n**KPSAligner** keypoint-based alignment, standard practice since 2015 (MTCNN, dlib, InsightFace).\n\nMeta's SFace variant seems to be scaled larger than the public reference (96 MB vs. ~40 MB; 2048-dimension output vs. the reference's 128–512).\n\nWorth stating plainly: **shipping detection and embedding models is not, by itself, evidence of recognition**. Plenty of apps run on-device face detection for framing or autofocus.\n\n## A cosine-similarity face index, dimensioned exactly to the on-device fingerprinter\n\nThe recognition pipeline that actually runs and reads into this database:\n\n```\n/data/user/0/com.facebook.stella/files/rldrive/person_profiles/objects.db\n```\n\nThis lives under **RLDrive**, Meta's cross-device sync framework, in a `person_profiles`\n\nnamespace designed to be populated remotely. I did **not** directly observe Meta pushing data to `person_profiles`\n\nspecifically on my test account. I want to be clear that I'm describing the channel's existence, not an observed transmission.\n\nThe schema:\n\n```\nCREATE TABLE person (\n  nodeid  INTEGER PRIMARY KEY,\n  name    TEXT,     \n  uri     TEXT,     \n  blob    BLOB,\n  deleted INTEGER,\n  version BLOB\n);\n\nCREATE TABLE face (\n  nodeid    INTEGER PRIMARY KEY,\n  mediaPath TEXT,    -- the face_id used in the deep link\n  personUri TEXT,    -- soft reference back to person.uri\n  blob      BLOB,\n  deleted   INTEGER,\n  uri       TEXT,\n  version   BLOB\n);\n\nCREATE VIRTUAL TABLE face_mediaPath_vec\n  USING vec0(mediaPath float[2048] distance_metric=cosine);\n  -- 2048-float biometric fingerprint per face, cosine-distance search\n  -- (uses the sqlite-vec extension)\n```\n\nEach `face`\n\nrow points at a `person`\n\nvia `personUri`\n\n. Each `face.mediaPath`\n\nis the primary key into `face_mediaPath_vec`\n\n, which stores the 2048-number embedding. Recognition is a cosine-similarity query against that index, followed by a join into `person.name`\n\nfor the notification text.\n\nA few things line up:\n\n`vec0`\n\nis the open-source**sqlite-vec** extension, which turns SQLite into a vector-similarity engine.- The dimension\n`float[2048]`\n\nis the exact output shape of the SFace embedder shipped on the app. - The\n`cosine`\n\nmetric is the standard choice for comparing face embeddings.\n\nThe schema permits multiple `face`\n\nrows per `personUri`\n\n(no `UNIQUE`\n\nconstraint), but whether a production deployment uses one-to-one or one-to-many is not visible from a non-enrolled device.\n\n**End-to-end test confirms both branches and isolates where writes go.** I SHA-256-snapshotted and row-counted the database, then ran the full recognition pipeline twice: once against an empty index (no-match), once against an index pre-loaded with a single embedding (match):\n\n**No match**(empty`face_mediaPath_vec`\n\n): one`(uuid.jpg, uuid.emb)`\n\npair was written to`NameTagsPending/`\n\n. No notification.**Match**: an Android notification fired through the production`nametags_recognition`\n\nchannel - title*\"Person recognized\"*, body*\"Recognized Michel Foucault\"*. Nothing was added to`NameTagsPending/`\n\n.\n\n## Unrecognized faces are staged to disk: crop plus fingerprint in `NameTagsPending/`\n\nWhen the device sees a face that the local index does not match, Stella writes it to:\n\n```\n/data/user/0/com.facebook.stella/files/NameTagsPending/\n```\n\nEach unrecognized face produces a pair of files named with a fresh UUID:\n\n- a\n`.jpg`\n\n— the cropped, aligned face, the output of SCRFD + KPSAligner; and - an\n`.emb`\n\n— the 2048-number SFace fingerprint.\n\nThe directory is mode `0700`\n\nand survives reboots. Writes happen **only** on the no-match branch; matched faces go to a notification and leave no on-disk trace.\n\nI verified the embedding's structure directly:\n\n```\nFile:    NameTagsPending/1566ab46-[...].emb\nSize:    8,192 bytes (2048 × float32, big-endian)\nL2 norm: 0.999999          ← canonical L2-normalized face embedding\nMin/max: −0.092110 / +0.098950\nMean:    +0.000292\n```\n\nTogether, `(uuid.jpg, uuid.emb)`\n\nis a complete, indexable biometric record of one face — the same shape and encoding the cosine index in `person_profiles/objects.db`\n\nis built to match against.\n\nThe name *NameTagsPending* most literal reading is \"faces pending a name\" — biometrically encoded, awaiting a label. I'll note the structural fact and let it carry its own weight: a face image and its fingerprint, stored side by side in plaintext, mode `0700`\n\n, surviving reboots, is precisely the dataset you would assemble if you intended to retroactively identify faces once a label arrives.\n\n## The notification surface is fully wired\n\nStella defines a dedicated Android notification channel\n\n```\nNotificationChannel{\n  id          = \"nametags_recognition\"\n  name        = \"NameTags recognition\"\n  description = \"Notifications for recognized NameTags connections\"\n  importance  = IMPORTANCE_HIGH      (heads-up + sound + badge)\n  sound       = system notification sound\n}\n```\n\nThe notification template is hardcoded in the recognition handler. Title is always *\"Person recognized\"*; body is always `\"Recognized \" + name`\n\n, where `name`\n\ncomes from the `person`\n\ntable in `person_profiles/objects.db`\n\n:\n\n```\nNotificationCompat.Builder(ctx, \"nametags_recognition\")\n  .setContentTitle(\"Person recognized\")\n  .setContentText(\"Recognized \" + matched_name)\n  .setAutoCancel(true)\n  .setContentIntent(\n    PendingIntent.getActivity(\n      ctx,\n      matched_name.hashCode(),\n      Intent.ACTION_VIEW with\n        Uri \"fb-viewapp://name_tags?face_id=\" + face_id,\n      FLAG_IMMUTABLE | FLAG_UPDATE_CURRENT))\n  .build()\n\nNotificationManagerCompat.notify(matched_name.hashCode(), notification)\n```\n\nThe notification is tappable: its `contentIntent`\n\nis a deep link of the form `fb-viewapp://name_tags?face_id=<face_id>`\n\n, a Meta-authored URL scheme meant to open a person-profile screen inside Stella.\n\nOne honest caveat: in v273, **I could not find that destination screen**. Tapping the notification routes Stella to its default tab, because the target Compose destination is absent from the navigation graph. The notification fires; the screen it points at isn't built into this release.\n\n## A user-facing \"Connections\" entry point exists in the APK\n\nStella v273 contains a widget rendering a card under a section header titled **\"Connections\"**, with the text *\"See your connections\"* / *\"Remember the people you met and make new connections.\"* Both strings are hardcoded literals in the APK not server-pushed.\n\nOn a stock, unenrolled account, the card does **not appear** on the Glasses tab at all. It became visible during testing. In normal use, a user would not see this.\n\n## What this adds up to\n\n- The full on-device face-recognition stack: detection, alignment, embedding, vector index, storage, write path, and notification surface is present and assembled in Stella v273.\n- It is functional. Run end-to-end, it recognizes a known face and names it in a notification, and it stages unknown faces (crop + fingerprint) to disk.\n- The index dimension, embedding shape, and storage schema are mutually consistent, this is a coherent system, not stray dead code.\n- The pieces a user would actually touch: the \"Connections\" card and the profile screen the notification opens are either absent from the build or buried deeper.\n- The database the live pipeline uses sits in a sync namespace Meta populates server-side, alongside other namespaces it already populates, but I did not observe a push to the face namespace on my account.\n\nWhat I am **not** claiming: that Meta is identifying strangers for users today, that enrollment data is flowing, or that any of this is enabled in production.\n\nWhat's hard to wave away: building, shipping, and wiring this much apparatus down to an 2048-dimension facial fingerprinting and a hardcoded \"Person recognized\" notification, is an engineering investment. Capability that doesn't ship by accident. Whether and when it goes into production is Meta's to answer.\n\n*This research is published alongside reporting in WIRED.*", "url": "https://wpnews.pro/news/meta-s-ships-facial-recognition-on-smart-glasses", "canonical_source": "https://www.buchodi.com/meta-glasses-facial-recognition/", "published_at": "2026-06-04 19:36:48+00:00", "updated_at": "2026-06-04 19:49:40.391653+00:00", "lang": "en", "topics": ["computer-vision", "artificial-intelligence", "ai-products", "ai-ethics", "ai-research"], "entities": ["Meta", "Stella", "Android"], "alternates": {"html": "https://wpnews.pro/news/meta-s-ships-facial-recognition-on-smart-glasses", "markdown": "https://wpnews.pro/news/meta-s-ships-facial-recognition-on-smart-glasses.md", "text": "https://wpnews.pro/news/meta-s-ships-facial-recognition-on-smart-glasses.txt", "jsonld": "https://wpnews.pro/news/meta-s-ships-facial-recognition-on-smart-glasses.jsonld"}}