Context Engineering - 100x Claude Code Effectiveness With Obsidian
Your notes are not just notes anymore, they are the context layer your AI needs to think with you.
I’ve written about Prompt Engineering, the skill of crafting a prompt that guides AI to provide you with high quality and relevant answers. But even a perfectly crafted prompt falls short when the answer comes back generic, not tailored to your specific situation. As you get better at prompting, the bottleneck shifts from how you ask to what the AI actually knows about you and your situation. And even when you cram all the relevant background into the prompt, it doesn’t survive from one session to the next — so you provide it over and over again. Prompts are disposable.
I’ve written an article about using Obsidian with Claude Code (here). I’ve been using Obsidian for years as a tool to keep notes and organize my thinking. Daily notes have always been the core tool of capturing my daily activity, and my zettelkasten was my tool for organizing my thinking.
When I started combining Claude Code with Obsidian, I started to see the power of AI to help with my task management, harvest daily knowledge and wisdom, and iterate on making my systems better. I was just trying to get organized and take better notes and maintain my personal knowledge management system, and wound up with a way for AI to help me in a very personalized and impactful way by being extremely knowledgeable about my life, my projects, my ideas, and my thoughts. It was able to cross-link and synthesize across all of it. The context I could provide it was a powerful lever for getting better results.
Another time that shifted the way I worked with AI happened at my day job. We were building out our production infrastructure using a system called Terraform. It’s an infrastructure-as-code tool that allows you to design infrastructure using documents (code). You can write files that describe your firewalls, routers, databases, networks, and all the things that go into building the infrastructure to run our software. The benefit we were seeking was the ability to save this “infrastructure as code” in files that we could then version control and keep history about how we evolved it. It allowed a distributed team to see changes made to the environment and reason about it. And best of all, Claude knew the format well and could design, create and test this infrastructure code.
On one particular day, we were required to answer a whole lot of questions about our infrastructure and how it was set up in order to satisfy the need for GDPR compliance. This would normally take someone weeks of work to investigate actual configuration and setup of the network and environment. Instead, I fed the Terraform files and the infrastructure design documents to Claude Code along with the GDPR compliance questions. Claude was able to create a concrete response to each question and show where we were compliant and where we fell short. It then created a prioritized list of things we would need to change or implement. With this, I was able to create a compliance plan, execute it to implement the needed changes, and provided a robust and complete compliance report with proof of compliance by the end of the day. Work that would have taken a month done in 6 hours because we had the right information in a format that AI could use. The right context.
That was the same move I did with Obsidian. What I’d been doing with notes was what I’d done with infrastructure. Both are structured context that compound when combined with the power of AI. Having the context of our infrastructure made all the difference.
What I’d been doing all along was context engineering. I’d been doing it for months before I had a word for it. You probably have too.
A more effective way forward, once you hit the boundaries of what good Prompt Engineering can do is to start thinking about how to create a set of documents that contain the specific and custom knowledge about you and your situation that the AI can reference instead of you providing it over and over again. This knowledge grows in effectiveness over time as you add to it, refine it, and develop links between different documents and ideas.
Context Engineering moves the focus from what you ask, to what information you provide. It’s the second of five levels of working with AI I’ve written about — prompt engineering being the first.
Defining Context Engineering
What is Context Engineering? Here are some definitions that I’ve come across worth considering:
“The art of providing all the context for the task to be plausibly solvable by the LLM.” - Tobi Lütke (Shopify CEO, June 2025)
This definition uses the word “context” in the definition. But gets to the core simply.
“The delicate art and science of filling the context window with just the right information for the next step.” - Andrej Karpathy
This introduces that idea of “just enough” that will be important for us to consider later. And finally, a wonderful metaphor:
“Prompt engineering was about coming up with a magical sentence. Context engineering is about writing the full screenplay for the AI.” - Addy Osmani (Google engineer)
It’s the difference between the well known phrase “To be or not to be” and the entire script of Hamlet. Shakespeare might have described it thus:
What richer answer doth the engine give,
When fed not scraps, but all the writ I keep?
A single line may make the players live,
But hand them all the play, and meaning’s deep.
Claude Opus 4.8 when asked what Shakespeare might say about context engineering
Context Engineering is the discipline of deciding what goes into an AI’s context window, and possibly more importantly, what doesn’t. This is not about what words you pick to ask it a question, it’s about the entire information environment the model is operating inside of when it generates an answer. The quality of that environment determines the quality of what comes out.
Where Prompt Engineering improves the quality of the response when you ask a question, Context Engineering is all about what information you give to AI so it understands the specifics of your situation.
When done well, it will truly feel like the AI has gotten 100x better at working with you.
Personal Knowledge Management as Context
As I mention in a couple of my previous articles, your notes do more than help you remember things. Your notes are how your AI knows about you, what you care about, what you’ve done, and what you need to do. It’s where it knows about your projects and deadlines and who you work with and how. It is where you share enough about the why, what, and how of your work that it is able to create real impactful and specific work that you would never get just from a generic prompt, no matter how well engineered.
In previous articles (here and here), I talked about how I use Obsidian to take notes and use Claude to help me organize my day. This is an example of how your Personal Knowledge Management (PKM) system is ideal context for AI, if you maintain it well.
Here’s how:
Daily notes show the chronology of your life. This is your daily life context. What you did and thoughts you had throughout the day
Quarterly/monthly/weekly notes show your intention and priority that you are setting
TaskNotes show commitments and next actions.
Zettelkasten is your “second brain” and repository of synthesized thought
PARA (if you use Tiago Forte’s system of organizing notes) shows the scope and active/inactive boundaries of your projects and areas of responsibility
AGENT.md or CLAUDE.mddetermines persistent orientation for your AI to understand how to help you. It loads this every session as a default piece of context to use. The root of your Obsidian vaults should have an AGENT.md - the emerging standard, or a CLAUDE.md if you only use Claude Code.
All of these files are the Obsidian context that helps AI multiply its effectiveness when working with you on the ideas, topics, todos, and thoughts that you keep in your Obsidian (or any other plaintext note vault).
Maintaining some form of PKM notes repository is a really good idea to be able to provide your AI with context about what you think about, what you feel is important enough to write notes, the synthesis of your ideas and thoughts, and an interconnected web of connections between ideas. Here are a few ways of doing that.
The Knowledge Garden
A knowledge garden (also: digital garden) is a personal, living collection of notes that is:
Non-chronological — organized by connection and theme, not by when it was written
Exploratory — meant to be wandered through, not consumed once and archived
Perpetually unfinished — notes evolve in public/private rather than being “published”
Bidirectional — ideas link to each other, creating a web rather than a list
The key distinction: a knowledge garden is a state, not a stream. It reflects accumulated understanding at a point in time, rather than a sequence of moments. Notes aren’t just static entries — they’re living thoughts that grow over time if you tend to them.
Mike Caulfield (2015):
“Things in the Garden don’t collapse to a single set of relations or canonical sequence … every walk through the garden creates new paths, new meanings.”
Much of our life and information is consumed in streams, social media is a key example. Information is chronological, reactive, fleeting. The idea of tending to a Knowledge Garden is the opposite, it’s intentional, associative, and evolving. A knowledge garden is where you nurture ideas and thoughts and help them grow, evolve, emerge. It’s a slower, more intentional way to associate with the information in your life.
And the good news in the age of AI? AI is not great at consuming streams, it’s expensive and low fidelity. AI is great at reading, synthesizing, and using knowledge gardens.
Zettelkasten
A specific form of a knowledge garden that is popular right now is called a Zettelkasten. Zettelkasten means “slip box” or “note box”. It was famously used by Niklas Luhmann, who built a massive collection of notes on individual cards and linked them together based on theme, ideas, or other organizing principles that he thought were relevant. This system helped him to be prolific in his research and writing. Luhmann produced around 50 books and hundreds of articles from 90,000+ index cards.
Luhmann called his Zettelkasten:
“A kind of secondary memory will arise, an alter ego with whom we can constantly communicate.”
Recently, the Zettelkasten has been popularized by a book by Sönke Ahrens called “How to Take Smart Notes: One Simple Technique to Boost Writing, Learning and Thinking - for Students, Academics, and Nonfiction Book Writers (2017)”. If you are interested in this topic, I highly recommend reading a copy of this book, it has been foundational for how I think about PKM and how I take, link, and synthesize notes.
Modern AI researchers have embraced this: the A-MEM paper (NeurIPS 2025) directly applies Zettelkasten methodology to LLM agent memory, explicitly citing it as inspiration.
More is not necessarily better
“A context window is not memory. It is an expensive, lossy working surface.”
You may be thinking that if context is good, then “more must be better!” But research is discovering that this is not the case. Too much context can start to degrade the effectiveness of the context. The key to high performing Context Engineering is also about discrimination and deciding what NOT TO INCLUDE.
There is research showing that information that is buried in the middle of long prompts/context is routinely missed. So put your questions, directives, and things you want the LLM to act on reliably, at the very beginning or end of the prompt or context window.
Also, if you have a context repository that contains many drafts of different things, evolving decisions or understandings, or just not relevant, be aware that that information will compete with the relevant information and can start to corrupt or produce incorrect drift on the work done by AI because you are polluting the context window.
You need to choose what matters and strike a fine balance between too little and too much context.
Efficient Context Requires Structure
Good AI-readable knowledge has:
Clear headings (retrieval handles)
Semantic sections (each one coherent alone)
Source-of-truth documents
Metadata: date, owner, status, project, source
Links between related ideas
Review and deprecation practices
“A human can skim messy notes. An agent needs structure.”
If you use note taking (like Obsidian), and are fluent with markdown, then you are already providing much of this structure needed by AI to be efficient. Using headings (#, ##, etc.) in markdown and nesting semantic sections can really help reduce how many tokens are needed for AI to find the information it needs. Markdown headings are not just formatting anymore. In the age of AI, they are retrieval handles for machine collaborators. Properly structured headings that make semantic sense in their nesting can boost AI comprehension of a document with much less context use as many times it will not need to load the entire document into context.
I use a tool called jdocmunch to do just this, and it’s been super effective at making my document searches and processing much less expensive with reduced token usage. The heavy lifting is offloaded to CPU-bound code, rather than needing to feed the entire document to an LLM and use precious tokens and context space.
For coding, jcodemunch is a similar tool that indexes the syntax trees of code and allows for similar savings when AI reads and searches code.
Supercharge teams with shared context
We’ve been talking about PKM and personal context up until now. The next step is to think about the information architecture of shared team context. When you have a set of markdown files that both humans and AI share for work done across a team, the effectiveness of the whole team goes up and many of the paradigms for personal context still apply, but there are other considerations. The deeper details go beyond the scope of this article. For now, some high level thoughts on shared team context are below.
Meeting notes can share update context that AI can help summarize for humans as well as update agreements, documents, and issues with decisions made. Shared proposals, designs, architecture documents, and specifications help the team have a shared understanding of what is being collaboratively created together. Misunderstandings go down, and shared understanding goes up.
I work with teams that use GitHub as that shared repository, but you could also use Notion, Google docs and spreadsheets, or any system of sharable information. The key is that you are able to natively access it via AI, or have an MCP (Model Context Protocol) server available for AI to interact with.
Having a shared context space with your teams is a powerful way to sync, stay aligned, and to work better, faster and with more agency due to the power of shared context. More on that in a future article.
Context Is the New Literacy
The future will not belong only to the people with access to the best models. The models will keep changing. The future will belong to the people and organizations who can make their knowledge legible, current, retrievable, and alive.
Context engineering starts simply: write things down. Structure them. Keep them current. Make them available to the machines that are now helping you think.
Your notes are not just notes anymore. They are the world your AI wakes up inside.