Our Quest for Meeting Summary Accuracy: What We Learned Building Meetily

When we started building Meetily, our north star was privacy. No cloud uploads. No meeting bots. Everything runs on your machine.

We assumed the AI models had caught up. Whisper for transcription. Local LLMs for summarization. Simple.

We were partially right about transcription. But summarization? That's where things got complicated.

The Promise We Made

Meetily was built on a clear principle: 100% local processing. Your meetings contain sensitive information - business strategies, client details, personnel discussions. We believed (and still believe) that you shouldn't have to trust a third party with this data.

So we shipped the Community Edition with local-only AI:

• Transcription: Whisper.cpp running entirely on your device
• Summarization: Local LLMs via Ollama or our bundled Gemma models

It worked. But "works" and "works well" are different things.

What We Heard From Users

Within weeks of our release, the feedback started coming in. The transcription was solid - Whisper is genuinely impressive. But the summaries? Users reported:

• Hallucinations - The AI would confidently state things that were never said
• Missing action items - Key tasks assigned in the meeting didn't appear in the summary
• Missing important points - Major decisions glossed over or omitted entirely
• Context issues - The AI would misunderstand who said what, or conflate different topics

For a product that promises to capture your meetings accurately, this was a problem.

Why Local Models Struggle

We tested several local models:

The issue isn't that local models are bad. It's that meeting transcripts present a specific challenge:

1. Long context - A 1-hour meeting might generate 10,000+ words of transcript
2. Implicit context - "Let's do what we discussed last time" requires understanding what wasn't said
3. Speaker attribution - Who said what matters enormously for action items
4. Domain knowledge - Technical jargon, company-specific terms, acronyms

Smaller models (under 7B parameters) simply don't have the capacity to handle all of this reliably. And larger models that can? They need serious hardware - we're talking 32GB+ RAM, ideally with GPU acceleration.

Most users don't have that. And even those who do often found the processing time unacceptable.

How Meetily's Pluggable Architecture Helps

From day one, Meetily was designed with a pluggable model interface. Users can connect:

• Local models: Ollama, bundled Gemma, any OpenAI-compatible local endpoint
• Cloud models: Claude, OpenAI, Groq, or any OpenAI-compatible API
• Hosted AI: Our own hosted option. No API keys, no setup. Just works. Pro users get generous free credits, and trial users get 10 free credits to try it out.

This wasn't a pivot - it was always the architecture. What we learned from user feedback is that power users who could run both options typically preferred cloud models for summarization quality, while keeping transcription fully local.

We also heard from users who just wanted something that worked out of the box without setting up API keys or running local models. That's why we added Hosted AI. You don't need to configure anything. Start a meeting, get a summary. If you don't like it, switch to local or BYOK anytime.

Some users genuinely cannot use cloud services. Healthcare providers with PHI. Legal teams with privileged communications. Security researchers. For them, the fully local path works well.

For users whose meetings don't contain regulated data, a cloud model with a clear privacy policy is often the right trade-off for better accuracy. The point is: you get to choose.

The Providers We Support (And Their Privacy Policies)

When we decided to add cloud options, we didn't just pick the most popular providers. We read every privacy policy, data processing agreement, and terms of service. Here's what we found:

Anthropic Claude API

Claude consistently produces the highest quality meeting summaries in our testing. Anthropic has been clear: API inputs and outputs are not used for model training. After September 2025, they reduced retention from 30 days to 7 days.

For users who can accept cloud processing, Claude is our recommended option for accuracy.

Source: Anthropic Privacy Center(opens in new tab)

Groq API

Groq offers a variety of models (Llama, Mixtral, Gemma, and others) at extremely fast inference speeds via their custom LPU hardware. Their default is to not retain customer data, and you can explicitly enable Zero Data Retention.

For users who want cloud speed without prolonged data storage, Groq is a solid choice.

Source: Groq Privacy Policy(opens in new tab), GroqCloud Data Documentation(opens in new tab)

OpenAI API

OpenAI's API has a well-documented privacy stance: they do not train on API data unless you explicitly opt in. Zero Data Retention is available for enterprise customers.

Source: OpenAI Enterprise Privacy(opens in new tab), OpenAI Business Data(opens in new tab)

Google Gemini API

Gemini is where you need to read the fine print. Free tier users have their data retained for 55 days AND used for model training. Paid and enterprise customers get shorter 30-day retention and are excluded from training.

If you use Gemini through Meetily, understand which tier you're on.

Source: Gemini API Terms of Service(opens in new tab)

OpenWebUI (Self-Hosted)

OpenWebUI is a self-hosted interface for running LLMs. When configured properly, it gives you the best of both worlds: better models than you can run locally, with complete data control.

For privacy-maximizing users who have the technical capability, this is the gold standard.

Source: OpenWebUI(opens in new tab)

Local Ollama Models

For users who cannot accept any cloud processing, Ollama remains available. We bundle Gemma models for users who don't want to install Ollama separately.

Quality varies by model and hardware. If you have a powerful machine (32GB+ RAM, GPU), Mistral 7B or larger models can produce good results. On modest hardware, expect trade-offs.

Our Current Recommendation

Based on our testing and privacy research:

What We're Still Figuring Out

This isn't a "we solved it" post. We're still exploring:

1. Fine-tuning local models - Can we train a smaller model specifically for meeting summarization?
2. Hybrid approaches - Local processing for sensitive sections, cloud for general summary?
3. Better prompting - Are there prompt engineering techniques that improve local model output?
4. Newer models - The local AI landscape evolves rapidly. Today's limitation might be solved in 6 months.

We committed to building a privacy-first meeting assistant. That commitment doesn't mean we pretend local models are perfect when they're not. It means we're honest about trade-offs, transparent about provider policies, and committed to improving.

The Transcription Side: A Brighter Story

While summarization has been challenging, transcription is working well locally. We offer two engines:

Whisper (via whisper-rs)

• Models: tiny, base, small, medium, large-v3
• Languages: 99+ supported
• Quality: High accuracy with large-v3
• Hardware: Scales with model size; large-v3 needs ~3GB VRAM

Parakeet

• Optimized for: English and French
• Quality: Excellent for supported languages
• Speed: Faster than equivalent Whisper models
• Best for: Users who primarily have English/French meetings

Both run entirely locally. Both work offline after initial model download. For transcription, the local-first approach works.

The Trade-offs Are Real

"Local-first" doesn't automatically mean "best." Sometimes it does. Sometimes it means trading quality for privacy. Here's how we think about it:

• For transcription: Local models (Whisper, Parakeet) work excellently. No reason to go cloud.
• For summarization: Quality gap is real. Smaller local models hallucinate more. Larger models need serious hardware.
• Your choice: Meetily lets you pick - fully local, fully cloud, or mix and match per use case.

What This Means For You

If you're using Meetily:

• Community Edition offers fully local processing
• Pro Edition includes Hosted AI with free credits, plus cloud model integrations
• You choose which provider (if any) to use for summarization
• Privacy policies above help you evaluate the trade-offs

The architecture was always designed for choice. Use what works for your situation.

Try It Yourself

Meetily Community Edition is free and open source. Pro includes Hosted AI with free credits for users who just want great summaries without setup. Or bring your own API key. Or go fully local. Your call.

Whether you go fully local, use Hosted AI, or connect your own cloud provider, you're in control.

This post reflects our understanding as of February 2026. Privacy policies change. We'll update this post if providers make significant changes. If you spot anything outdated, email us at meetily@zackriya.com.