Category: Uncategorized

A year ago, we were just getting comfortable with prompt engineering.  Vibe coding wasn’t yet common parlance.  A lot changes in a year.  Today, Copilot is an ever-present companion for many of us, and getting the most out of it is an ever-evolving part of the practice of development.

Just as effective prompt engineering makes a big difference in the quality of output you can get from an LLM, the right guidance for Copilot will help it produce better work - especially in agent mode.  This article will focus on creation and use of Copilot-instructions to guide operation on a repo-by-repo basis, but before diving in, let’s review a few options in this ever-evolving area.

Instructing Copilot

The principle difference between prompt engineering and Copilot instructions is that prompt engineering tends to be transient and is certainly localized to your account-specific context. 

It’s true that Copilot Memory is attempting to fill this gap, and given emerging tools like OpenClaw, the transient nature of prompt engineering seems sure to disappear, but instructions that stick to you individually won’t maintain consistency in a team setting.

Enter Copilot-instructions, which let you save your project-wide Copilot instructions in a file that lives with your repo, visible to anyone who pulls your repo.  We’ll explore this option in depth shortly, but keep in mind this is not the only option to capture design and coding guidance for AI coding tools.

For guidance that can extend across multiple repositories, consider MCP integration with your project organization tool of choice.  Atlassian offers an MCP server for its cloud-hosted Jira and Confluence, and Microsoft has a similar MCP server for AZDO, as well as a Copilot connector.  Of these tools, the only one I’ve tried personally is the Atlassian MCP server, which operates mostly as-advertised.  Note that when used properly, the MCP integrations above can guide programming standards across all your code bases all at once, which is the stuff of dreams for enterprise architects - as long as these standards can be expressed efficiently and effectively.  It’s also worth pointing out that all these techniques can coexist, with Copilot-instructions operating in a bootstrapping manner to access enterprise-wide standards.

Getting Started

The basics for Copilot-instructions are as simple as could be - simply drop a file called Copilot-instructions.md in a .github folder in your repo.  Docs from github will go into detail on options like instructions that apply only to some paths or some agent LLMs.  This page also offers a prompt you can use to generate a Copilot-instructions file based on your existing repo, and that’s what we’ll try here.

For this example, I’m using a simple solution I created to explore vertical-slice architecture in dotnet Aspire.  You can find the “before” version of this repo here:  https://github.com/dlambert-personal/AspireTodo

For this demo, I’m going to use the prompt shown in the github docs verbatim.  I do believe that were I doing this in earnest, I’d want to customize the prompt somewhat.  I may also want to explore how this prompt works against repos of varying size, maturity, and cohesiveness. For instance, the bit where we ask Copilot to derive architectural elements of the project is especially interesting.  In a trivial example like this, I expect Copilot to find those elements quite effectively, but in a larger repo – one with some development history – I wouldn’t be surprised to see the model struggle a bit.  For reference, I’m using the Claude-Haiku 4.5 model in agent mode.

Reviewing the Results

So, how did the prompt do in its out-of-the box form?  Really not too bad; there’s some great content here.  My chief criticism (and one that’s easily corrected) is that I believe much of the generated content would do well migrated to the readme.md file, which I’ll do when I commit the “after” results to a new repo.  So, section-by-section, here’s what was produced.

• Common Tasks & Gotchas.  This section began to get a bit odd.  There’s a bit on adding a new API endpoint that’s pretty good, but there’s also a bit on debugging JSON deserialization that seems like pretty general debugging techniques I’d expect to uncover when needed in Ask mode.
• Known Issues & Workarounds.  This entire section suffers from the same problem - this entire section should be discoverable when needed and debugging topics shouldn’t be limited to the few topics presented in this section.

Putting it to the test

Before I fine-tune the instructions, I wanted to test Copilot to see if it seemed any smarter with the addition of the initial draft.  I made a copy of the original repo (so you can see the “before” version) and asked Copilot (still using Claude 4.5) the following:

This solution needs to be refactored to account for a name change.  Replace all references to AspireTodo with AspireTickets.  Include solution name, project names, namespaces, component files, and variable names.  As always, the solution should build successfully when you're done.

Copilot plotted a six-step plan and got stuck on step 2:

This refactoring is extensive. Let me take a systematic approach:

1.    First, I'll gather all the source files that need namespace updates
2. Rename directories and projects
3.   Update all namespace declarations
4.    Update project references
5.    Verify the build succeeds

This is a problem I’ve seen frequently with Copilot: the powershell integration has a tendency to fail or freeze.  After closing the terminal, Copilot realized it needed another approach, and it tried again.  This went on for a while - the whole time showing success sprinkled with failure.  Had this been an actual developer, I’d have stopped them and suggested they revisit their plan, as it sure looks like a lot of guessing going on here.  The “stuck in powershell” problem continued as well – it seemed like Copilot expected powershell to understand commands that powershell just whiffed on - despite my installation being reasonably current.  

Finally, adding insult to injury, Copilot eventually just stopped working new tasks - apparently before it was done.  I gave it a little nudge, and sure enough - there was a little more work to do.

Finally - the build.  First attempt: five errors and two warnings.  All five errors were project files not found.  Copilot blamed them on Visual Studio’s cache, which seemed like a class-one cop-out. 

More confusion… more freezes.  Eventually, Copilot froze hard enough I couldn’t even cancel in the chat window, so I took matters into my own hands.  It turns out that aside from the file renaming issues, Copilot had gotten pretty close.  I had one “using” statement I needed to patch, and all the project files had to be removed and re-added, but once that was complete, I was back to building w/o errors again.

When all is said and done, did Copilot save me time in this refactor?  Maybe - but not by a ton.  I think the level of supervision is still quite high, but the remaining rough bits seem like they’re not so much a model shortcoming as an integration snafu - hopefully an easy update.

Next Challenge - adding to the UI

I suspected Copilot might do better with a change that didn’t require so many file manipulations, so I gave it a simple prompt that would require touching the solution in several places:

Modify the current page to be able add tickets in addition to displaying them.  Use the CreateTicket command in the API project to save the new ticket.

This request was dispatched quickly, and seems to have produced a high-quality output:

The project still built, and when I ran it, I saw a plausible-looking form.  The bad news: the form wasn’t submitting anything to the API, so I asked Copilot to fix it.  The resulting back & forth was almost comedic. I wasn’t about to let Copilot off the hook, and it did eventually figure it out.  I’d begun counting form submission attempts, and (if my count was right), it took well over two dozen stabs at this to get the form to submit successfully:

It could be argued that very little architectural guidance was provided to Copilot in the instructions we’d created, so the Blazor form submission stuff it struggled with might almost have been excusable.  I’m still a little disappointed here, though, because the real gotcha seems to have been an addition to .NET 10 (@rendermode) that was keeping the form from operating interactively, and the .NET version absolutely was covered in copilot-instructions.

For the last test / evaluation, I’m going to switch from Visual Studio 2026 to VS Code, and I’ll modify my prompt to specifically call out use of copilot-instructions, though I don’t believe this should be necessary.  This is the prompt used (including a typo on “architectural”):

Modify the solution to be able edit tickets that have been previously saved.  Use copilot-instructions.md as a reference to architecural norms in this solution.  Include UI changes to the Tickets page to load a ticket to the form for editing and submit changes to the API.  The API project will need a new EditTicket command to save the ticket.

I’ll be honest - I wasn’t expecting a drastic change in experience going from Visual Studio to VS Code, but wow, what a difference there was!  The initial change was quick, and as near as I can tell, perfectly functional.  I got an updated form that works as expected, the API adapter works, the API has been extended to include a new command, and the new command follows the design conventions in use by previous commands.  VS Code even asked me if I wanted it to test the new functionality, and that went slightly less-well – the spots where Copilot integrates to the world around it really do seem to be its Achilles heel right now.

Up to this last step, I believe I’d have given Copilot a barely-passing grade.  A co-worker from years back always used to remind me we don’t cheer for the dancing bear because it dances well – we cheer because bears aren’t supposed to dance at all, and this is about how I’d have summed up my experience with Copilot in Visual Studio.  Copilot in VS Code, on the other hand, seems much more well-sorted.

I’m still impressed with the amount of project documentation Copilot was able to sift out of my project, and I intend to press on with pulling some of the docs into readme and fine-tuning the bits that are left to be more prescriptive – in short, leaning on this a little more to get a feel for the edge cases.

Last year, my son introduced me to a podcast called “Black Box Down”.  Each episode is a discussion about an air disaster of some sort, and in the vast majority of cases, the eventual BOOM is preceded by a long sequence of small problems, dealt with poorly, or over-reacted-to until recovery is impossible.

Exceptions are like that.

I’ve been working on getting DataStax / Cassandra installed on a machine running on Hyper-V, and although it’s been a little while since I experienced the joy of Java stack traces, it’s coming back to me in a hurry.  I Just worked through one that turned out to be rooted in my Java version.  It turns out when the installation requirements indicated Java 8 or better, they left out, “but not too much better”.  After a bit of googling, I found a link suggesting I might need to roll back my Java version, and that did, in fact, make things better.  The process to diagnose has left a bit to be desired, though - starting with the epic stack trace..

What might have been better than this?  How about an assertion on startup to flag the Java version as a problem?  Once upon a time, this software was using a contemporary version of Java, and it would never have occurred to any reasonable developer to put an assertion like this in place, but at this point, it would be really helpful.  Do I have any reasonable expectation that this sort of assertion would make it into the source code?  Nope.  I fully expect that anyone expert enough in this package to contribute to it is well aware of the Java version required, so there’s just no benefit to them to include an assertion like this.  Next problem: packages built on packages built on packages, which make it pretty hard to pin down who should really be responsible for calling out the version dependency.

You may be thinking at this point that an IaC or container solution might be helpful, and I agree.  In this particular case, one of the alternatives at my disposal was an Oracle Virtual Box image that was already set up.  I opted out of that because I’ve already set up Hyper-V and Docker Desktop, and I’m concerned about having too many virtualization platforms sparring with one another.  The “bare metal” setup wasn’t ideal right out of the gates, but the silver lining in this case is that I’m learning a ton.

My biggest takeaway from this experience is going to be to “embrace the suck” – paying attention to the frustration and impact on productivity so I can watch for problems like this in software I’m producing.  As always, putting yourself in your users’ shoes will pay dividends, I believe.

Change is afoot once again. We've heard a ton about 5G for the last couple years, and we're just now starting to see this technology emerge in the market. Plenty of pundits have speculated that 5G is going to have the same magnitude of impact as the introduction of the web or the birth of mobile computing. Last week, we saw Facebook launch a major rebranding last week. The Metaverse is coming, powered in part by that same 5G network.

What's the connection to 5G? Edge computing and experiences. In this article from Ericsson, Peter Linder cites 5G as a key enabler for AR and VR without the use of a tethered PC. Few of us can imagine how our business applications will take advantage of experiences like this, but recall that just a few years ago, we'd have had a hard time imagining the rich mobile applications we take for granted today.

These new experiences aren't going to take the place of the experiences we've got today -- they're going to add to them. If you're not already seeing the same explosion in channels, that's coming, too. Partners, new brands, bundled products -- all these channels demand access to the capabilities you've got, packaged up in new ways.

There's no way to support this explosion without careful separation of experiences from capabilities. You should never see the mechanisms by which a command is carried out implemented in the same place your customer experiences it!

Spotify has some great experiences enabled by this sort of separation. I was streaming from my desktop PC this week through my "good" speakers (better than my laptop!), but I also had a Spotify window open on my laptop. I just happened to see the play list sync'ed to the laptop screen, and when I clicked next there, my session on my desktop followed right along. Although this is an ultra-simple example, it's evidence that Spotify is using its interfaces to send Commands to a back-end service -- nothing about the session that's streaming is connected to the commands at all -- otherwise, the audio source would have changed!

At this point, a little faith may be required in order to see a need like this in your enterprise, but for those with preparation on their side, I believe a myriad of new experiences will be possible in a few short years.

Last week, I was reminded of a lesson I learned from one of my mentors many years ago: when you're regression testing, "different" is all you really need to look for.

A little context would probably help, here.  Nearing the end of a sprint, we'd tested the new features in the sprint, but we really weren't sure whether we had good regression coverage.  Testing from prior sprints had part of the answer, but we couldn't really afford the time to run all the tests for this sprint and all the tests from our prior sprints.  Thus, the conversation turned to regression testing, automation, and tactics to speed it all up a bit.

The naive approach to regression testing, of course, is to run every test you've ever run all over again, checking each value, examining the dots on the i's and the crosses on the t's.  It's absolutely irrefutable and in the absence of automated tools, it's also completely impractical.  With automated tools, it's merely incredibly difficult, but fortunately, in most cases, there's a better way.

Segue back to my years-ago epiphany, in which I was struggling with a similar problem, and the aforementioned mentor gave me a well-aimed shove in the right direction.  He pointed out that once I'd achieved "accepted", and thus, "good", all I needed to look for when regression testing was "different".   All by itself, this didn't do much to help me, because looking for "different" still sounded like a whole lot of looking.   Combined with the fact that our application was able to produce some artifacts, however, this idea vaulted our testing forward.

Our application, it turned out, supported importing and exporting to and from excel files, so we were able to use this to radically speed up the search for differences -- we saved an export from a known good test, and future tests would just import this file, do some stuff, and export it back out again.  At this point, a simple file compare told us whether we'd produced the same output we'd previously validated as "good".

Armed with this technique, we began looking for other testable artifacts -- to the point where we built in some outputs that were used only for testing.  At the time, it was time well spent, because the maturity of testing tools made the alternatives pretty prohibitive.

And what do you do about a difference when you find one?  First reactions notwithstanding, it's not necessarily a bug.  A difference could arise as a result of an intended change, so it's not at all unusual to be able to attribute a difference to a change you're actually looking for; in which case, you throw out the old baseline and replace it with a new one.  Just remember you'll need to explain each difference every time you find one, which brings me to a final issue you'll want to sort out if you're going to use this technique.

Any artifact you want to use in this way must be completely idempotent -- that is, when you run through the same steps, you must produce the same output.  This seems like a gimme, but you'll find that a lot of naturally-occurring artifacts will have non-idempotent values like ID's, timestamps, machine names, and so on -- you'll need to address these in order to have artifacts you can use effectively for testing.

Once you get past this little hurdle, you're likely to find this idea powerful and super-easy to live with, and what more can you ask for in regression testing?