The Battle of the Green Dot

A BeZoned engineer built a presence detector that uses an old Philips monitor's built-in webcam and machine learning to automatically switch Microsoft Teams status between Available and Away, addressing complaints that Teams marks users Away whenever they stop typing or moving the mouse. The engineer first tested the webcam with ffmpeg, which reported the device supports 30fps modes from 176x144 up to 1920x1080, after an initial capture attempt failed because the default 29.970030 framerate was unsupported. The project targets a real workplace friction point: Teams' Away status is often misread as not working, even when someone is reading, sketching, or waiting on an AI agent.

Even if you've never used Microsoft Teams yourself, you've probably come across people complaining about it. There are plenty of complaints, but one seems to come up over and over again: 1. You're working as usual. Your Teams status is Available. All good. 2. You look away for ten seconds. 3. The status is Away. Your manager is writing you to ask why you are not working. Okay, I'm exaggerating. But the underlying problem is real. The fact you are not moving mouse or typing doesn't mean you are away. You could be: 1. reading an article, documentation or something else work-related 2. drawing, sketching or taking notes on paper 3. waiting for an AI agent to finish 4. dictating feedback or instructions to an AI agent But Microsoft Teams does not know that. Neither do your colleagues. Some teams even interpret Away status as "this person is not working". Not only is that assumption wrong, but it also puts unnecessary pressure on people to appear Available all the time. You'd be "away" in the office as well. You'd grab a coffee, walk over to a colleague to ask a question, or go to the bathroom. Nobody questions that in the office... Luckily, we see things differently at BeZoned https://bezoned.com?utm source=malja&utm source=blog&utm campaign=presence detector . We don't mind if you go Away during the day. We all work from home, and we trust each other. But trust doesn't make presence information useless. We still need to know when someone's around, especially when we're trying to have a quick conversation. When Teams switches to the Away status, but I'm still sitting at my desk, I'm sending the wrong signal. Others can still reach me. And it works the other way around, too. When I have to step away for a moment - to answer the door for a mailman, grab a food delivery because I'm too lazy to cook, or rescue my dog's toy from wherever he's managed to get it stuck - I tend to forget to set my status accordingly. That's when I thought: what if Teams could detect whether I'm sitting at my computer and automatically update my status? Sometimes you have the solution in front of you for years. In my case, it was an old Philips monitor with a built-in webcam. The webcam was terrible even when I bought the monitor, so I never used it for video calls. Maybe it was finally time to put it to use. Could a little bit of AI or rather ML help me turn this unused webcam into a presence detector ? When I'm sitting at the desk it would keep me Available on Teams. The second I move away, it would switch my status to Away. I could probably jump directly to the code, but I decided to start small. Let's take a picture with the webcam to see if it even works. First, I needed to list available video devices: ffmpeg -f avfoundation -list devices true -i "" Which outputs: AVFoundation indev @ 0x7513014140 AVFoundation video devices: AVFoundation indev @ 0x7513014140 0 USB2.0 FHD UVC WebCam The output is simplified, so it only contains the WebCam I'm interested in. Now let's take a picture: ffmpeg -f avfoundation -i "0:none" -frames:v 1 webcam.jpg And I'm greeted with an error: avfoundation @ 0x7676c20000 Selected framerate 29.970030 is not supported by the device. avfoundation @ 0x7676c20000 Supported modes: avfoundation @ 0x7676c20000 176x144@ 30.000030 30.000030 fps Last message repeated 1 times avfoundation @ 0x7676c20000 320x240@ 30.000030 30.000030 fps Last message repeated 1 times avfoundation @ 0x7676c20000 352x288@ 30.000030 30.000030 fps Last message repeated 1 times avfoundation @ 0x7676c20000 640x360@ 30.000030 30.000030 fps Last message repeated 1 times avfoundation @ 0x7676c20000 640x480@ 30.000030 30.000030 fps Last message repeated 1 times avfoundation @ 0x7676c20000 848x480@ 30.000030 30.000030 fps avfoundation @ 0x7676c20000 1280x720@ 30.000030 30.000030 fps avfoundation @ 0x7676c20000 1920x1080@ 30.000030 30.000030 fps in 0 @ 0x7676c1c000 Error opening input: Input/output error Error opening input file 0:none. Error opening input files: Input/output error After some fiddling, I got it working by explicitly setting the frame rate, resolution, and pixel format: ffmpeg -f avfoundation \ -framerate 30 \ -video size 1280x720 \ -pixel format nv12 \ -i "0:none" \ -vf "select=eq n\,60 " \ -frames:v 1 \ -y webcam.jpg And voila That's me. As I've said - it's a terrible webcam. On the other hand, you can clearly tell there is someone in the picture 😀 It should be more than enough for this use case. I started by passing the image to an LLM and asking it to detect whether there's a person in it. I used Apple' Foundation Models, available on macOS 27: fm respond --model system \ --image webcam.jpg \ "Is there a real person sitting at the desk? Answer only YES or NO." YES To make sure the model wasn't just answering "YES" to everything, I tried another picture. This time of an empty chair. And the response? NO Ta-da I could have stopped here. The model runs locally, it does not have any dependencies and running a bash script to do this would be easy. But Apple also provides Vision ML models, so naturally I had to test them as well. Vision models also support human pose detection, which could be used for gesture-based microphone controls, FaceTime-style reactions and other "very useful and impactful" features nobody asked for. I started with a simple Swift script, so I don't have to install anything. python import Foundation import Vision guard CommandLine.arguments.count 1 else { fputs "Usage: swift detect.swift image.jpg\n", stderr exit 1 } let url = URL fileURLWithPath: CommandLine.arguments 1 // Simple person detection without posture let request = VNDetectHumanRectanglesRequest // Checking just the upper body, because I'm sitting most of the time request.upperBodyOnly = true let handler = VNImageRequestHandler url: url, options: : do { try handler.perform request let people = request.results ?? print """ { "personDetected": \ people.count 0 , "count": \ people.count } """ } catch { fputs "Vision error: \ error \n", stderr exit 1 } And when executed: swift detect.swift webcam.jpg { "personDetected": true, "count": 1 } The first run was extremely slow, probably because Swift had to compile it. But subsequent detections were... well, swift . I also tried getting the detected person's bounding box and drawing a rectangle around them. The code got a bit longer, so I'll spare you the details. The result: Before getting too excited about detecting people with my webcam, I first checked whether I could actually update Teams status programmatically. Turns out, there are several ways to do it. If you've ever tried to address this issue yourself, you've probably come across mouse jigglers . They try to mimic user activity with more or less realistic mouse movement tricking MS Teams into thinking someone is actively using the computer. I wasn't particularly interested in moving my mouse around or pressing random keys just to keep Teams happy. Conveniently, we already have a desktop app that can update a user's Teams presence. As of now, BeZoned Sidebar https://bezoned.com/bezoned-sidebar-app?utm source=malja&utm source=blog&utm campaign=presence detector requires the user to click the Presence menu and set it manually. I could extend the Sidebar app to have this "presence detection" functionality baked in, or publish an API to other desktop apps. So anyone could extend the Sidebar functionality. If I were building this as a real product feature, integrating it would probably be the best option. But this was just a weekend experiment, and I was the only user. Modifying an app everyone else relies on is a no-go. What's left? Talking to the API directly. I could use the very same Graph API that BeZoned uses and set presence there. I'd need an Entra app registration, a few API permissions and some auth logic. But that would let me update my own Teams presence without touching the BeZoned app or affecting anyone else. I decided to stick with Swift. It's already available on my Mac and I wanted to play with it for a long time. Additionally, it's easy to share the codebase with my colleagues if they ever want to run it as well. When the script starts, it authenticates through Entra using the Device code https://learn.microsoft.com/en-us/entra/identity/conditional-access/concept-authentication-flows device-code-flow flow. I just open the auth page, enter the displayed code and sign in. A deeper integration with Keychain to preserve credentials is a problem for future me. To avoid constantly switching between Available and Away, I settled on a simple state machine : In short, the script requires two consecutive detections of the same result to set the status to Away or Available in Teams. The final piece was connecting everything to Teams. I used three endpoints: - setPresence https://learn.microsoft.com/en-us/graph/api/presence-setpresence?view=graph-rest-1.0&tabs=http updates the presence using a session ID, which is later used to clear the presence override - getPresence https://learn.microsoft.com/en-us/graph/api/presence-get?view=graph-rest-1.0&tabs=http returns the current presence - I run this before every "switch" to ensure I'm not just hammering API with unnecessary "Change from Away to Away" requests. - and finally clearPresence https://learn.microsoft.com/en-us/graph/api/presence-clearpresence?view=graph-rest-1.0&tabs=http clears the presence configured from this script if I terminate it or when I lock the screen. Teams is then free to set it according to normal rules. Just one delegated permission is needed for all three of them: Presence.ReadWrite And that's it. When I start my computer, the Swift script launches, wakes up the old webcam and starts checking whether I'm at my desk. My colleagues are updated in real time. Away finally means I'm actually away. About the author - I've worked remotely for most of my software engineering career. It opened up a whole new world to me. I was able to build my own office, cut out the commute, avoid distractions, and... wear the same shirt for a week. : Of course, there are downsides, but to me, remote work makes sense for both companies and employees. I've always aimed to build remote-first teams because I believe it leads to better documentation, smoother processes, and fewer bottlenecks. But there's always been one thing missing: that sense of belonging, of truly feeling like part of a team. So when I got the chance to help BeZoned build the best online office for remote teams, it was a dream come true. Not only did the idea immediately click with me, but I was also getting to scratch my own itch. Comments Join the discussion https://hashnode.com/posts/the-battle-of-the-green-dot/6ae3670695963ad3dbfaa7c9 No comments yet. Be the first to comment. https://hashnode.com/posts/the-battle-of-the-green-dot/6ae3670695963ad3dbfaa7c9 BeZoned Behind the scenes of building BeZoned: architecture, Microsoft Teams and Graph, product trade-offs, technical constraints, experiments, failures, and lessons from shipping. Along the way, I explore remote work, distributed teams, communication, digital presence, and the systems that make online collaboration actually work. Up nextWhy we chose Teams for BeZonedAbout BeZoned Before we start, allow me a short detour. I think I owe you a short description of what BeZoned actually is. Without it, the reasoning would likely not make sense to you. If you already https://malja.dev/why-we-chose-teams-for-bezoned