Markov

The race to build computer-use AI—agents that can click, type, and navigate software like a human—has mostly been a story about models. But models are only as good as the data they train on, and for computer-use, that data is notoriously scarce. Markov, a young company from Y Combinator's current batch, is betting that the real bottleneck isn't architecture but the raw material: recordings of how people actually use computers. With 150K+ downloads across its HuggingFace datasets, Markov is positioning itself as the data layer for this emerging category—not by selling access, but by giving it away.

The data bottleneck in computer-use AI

Computer-use AI is different from chatbots or image generators. It doesn't just need to understand text or pixels; it needs to understand the temporal sequence of actions—where to move the mouse, when to click, how to type—that accomplish a task. This is a behavioral problem, and it requires behavioral data. Yet most public datasets for computer-use tasks are either synthetic or captured in controlled environments, which don't reflect the messy reality of real software: laggy UIs, unexpected pop-ups, multiple monitors.

Markov's thesis is that the only way to train agents that can handle this mess is to show them the mess. The company records real people using real applications—Salesforce, Blender, Photoshop, even games like Valorant and Minecraft—and turns those recordings into structured training data. This is not a new idea, but the scale and the focus on complete workflows are. The site highlights 10,000+ hours of computer-use recordings and 500+ hours of gaming data, all open-sourced on HuggingFace.

What Markov actually captures and why it matters

A screen recording alone is not enough. Markov's datasets include synchronized mouse and keyboard events, narrations, and—for its CAD environments—problem statements, input references, gold output files, and rubrics. This is a significant step beyond raw logs. The inclusion of narrations is particularly interesting: it means the data captures not just what a user did, but why they did it, which is crucial for training models to make decisions, not just mimic actions.

The CAD environments dataset is a differentiator. CAD software like AutoCAD, SolidWorks, and Revit is complex, with steep learning curves and high-stakes outputs. Training an agent to operate these tools could unlock huge productivity gains in engineering and design, but it requires data that is both precise and contextual. Markov's approach—pairing screen recordings with gold output files and rubrics—suggests a focus on verifiable outcomes, not just action sequences. This is a smart bet: it's one thing to train an agent to move a mouse, another to train it to produce a correct engineering drawing.

The open-source strategy: 150K downloads as a wedge

Markov's decision to open-source its datasets is a classic data-layer play. By giving away high-quality data, the company builds brand recognition and community trust, while simultaneously setting the standard for what computer-use data should look like. The 150K+ downloads on HuggingFace are a testament to the demand—and a powerful marketing signal. For a startup with no disclosed funding, this is a low-cost way to establish credibility.

The open-source work also serves as a research portfolio. The website features a research section with detailed samples, which positions Markov as a serious technical player, not just a data vendor. This is likely to attract both potential customers (who need data for their own models) and potential collaborators (who might contribute to the datasets).

The risk is that open-sourcing the data could undermine future sales. If the open datasets are good enough, why would anyone pay for more? Markov's answer seems to be that the open data is the appetizer, and the full meal—custom datasets, enterprise support, bespoke environments—is what they're really selling. The site's call-to-action is "Talk to us," which suggests a consultative sales model rather than a self-serve storefront.

The hard problem: verifiable outcomes in messy workflows

One of the biggest challenges in computer-use data is verification. How do you know if an agent's actions actually achieved the goal? Markov's datasets address this in two ways: for CAD, they provide gold output files and rubrics; for other workflows, they rely on the completeness of the recording (start to finish) and the narration. This is a step in the right direction, but it's not a silver bullet.

For example, in a coding task, the "correct" outcome might be a working function, but there are many ways to write it. A rubric can capture some of this, but not all. Similarly, in a game like Minecraft, the goal might be to build a house, but the quality of the house is subjective. Markov's data is rich, but it's not clear how they handle these ambiguities at scale.

This is where the company's focus on reinforcement learning (RL) comes in. RL requires a reward signal, and verifiable outcomes are essential for computing rewards. By providing gold outputs and rubrics, Markov is enabling RL training pipelines that would otherwise be impossible. This is a strategic alignment: the data is designed not just for supervised fine-tuning (SFT) but for RL, which is where the future of agent training lies.

Naming the invisible layer: Markov's brand as a signal

The name "Markov" is a deliberate nod to the mathematical foundations of sequential decision-making. Markov chains and Markov decision processes are the backbone of reinforcement learning, and by adopting this name, the company is signaling its technical orientation. It's a name that resonates with AI researchers and practitioners, but it's also abstract—it doesn't say "computer-use data" on the tin.

This abstraction is a double-edged sword. On one hand, it gives Markov room to expand beyond its current focus. If the company eventually offers data for other types of agentic AI, the name still fits. On the other hand, it makes the company's value proposition less immediately clear to non-technical buyers. The tagline "Data for computer-use AI" helps, but the name itself requires explanation.

The domain, markovstudios.com, is a bit of a mouthful, but it's memorable and distinct. The use of "studios" suggests a creative, production-oriented approach, which aligns with the company's focus on capturing real workflows. Overall, the brand is well-suited for a technical audience, but it may need to evolve as the company grows.

Open questions: quality, scale, and the road ahead

Markov's approach is promising, but there are open questions. First, the quality of the data is only as good as the recording process. How are the recordings curated? Are they filtered for quality? The site doesn't disclose this. Second, scale is a challenge. 10,000 hours is impressive, but it's a drop in the bucket compared to the scale of text and image data used to train large models. To become the default data layer, Markov will need to grow its datasets significantly.

Third, the business model is unproven. The site doesn't disclose pricing or funding, and the open-source strategy could make it hard to monetize. However, the company's focus on enterprise-grade CAD environments suggests a target market with deep pockets—engineering firms, design software companies, and AI labs building specialized agents.

For now, Markov is a company to watch. It's tackling a real problem with a thoughtful approach, and its open-source contributions are already making a mark. If it can maintain quality while scaling, it could become the go-to source for computer-use data—and the name "Markov" might just become synonymous with the category.