I wrote a small CLI tool in Rust called fts_autosln. This tool generates Visual Studio a .sln a .vcxproj for an arbitrary application from its .pdb files (debug symbols). This is a very niche tool. My workflow is niche and I desperately needed this, so I built it.

This blog post will cover both what it does and how. The technical details may be relevant to other tool makers.

• GitHub: source code
• GitHub: binaries

Quick Example

Here's a quick example before we dig into gory details.

Running fts_autosln for UnrealEditor.exe the will generate UnrealEditor.sln, UnrealEditor_source_code.vcxproj, and UnrealEditor_source_code.vcxproj.filters. In Visual Studio it looks like this:

There's a few things to note:

• fts_autosln generates three files (.sln, .vcxproj, .vcxproj.filters)
• F5 will launch the executable
• There is no build support
• One vcxproj contains all source files
• All .h files are dumped into a single "headers" filter
• All .cpp files are stored in a flat list under their pdb

My expectation is that users rarely use the solution browser. Code is instead navigated via tools like Fast Find or 10x.

My Problem: Mixed Build Systems

Why is this useful? In a word, mixed build systems.

If you have one build system to rule them all you probably don't need this. Modern build systems already generate Visual Studio solutions.

Unfortunately I live in a world where I have mixed build systems. I write C++ code that is compiled via Buck to produce .dll plugins. Then I use those .dlls and header files inside an Unreal Engine project which is built with Epic's extremely custom Unreal Build Tool.

I have two build systems, but I have full source code for both sides. It is extremely convenient to have all source files in a single place. Making a new build may require invoking two systems. But I want to easily search, edit, and debug across both sides.

fts_autosln is also useful if your build system does not produce a .sln, or if it produces a bloated .sln with a kajillion third-party library files you'd like to ignore.

Detailed Usage

Let's go over some example usage. Imagine you have an application called foo.exe. You can generate foo.sln in one of two ways.

// from-file: recursively find and read pdbs from disk
fts_autosln.exe --sln-path foo.sln -source_roots "C:/path/to/src" from-file "C:/path/to/bin/foo.exe"

// from-process-name: load symbols via SymInitialize
fts_autosln.exe --sln-path foo.sln -source_roots "C:/path/to/src" from-process-name foo.exe
          

The pseudo-code for from-file looks roughly like this:

• run for foo.exe
  • find all relevant pdbs
    • open foo.pdb for foo.exe
    • extract what .dlls foo.pdb depends on
    • make best guess where each .dll would load from
    • recursively determine all dll deps and all pdbs
  • foreach pdb
    • extract list of source files
  • foreach source file
    • resolve local file path
    • note: pdb may have come from build machine and local path may differ
  • generate .sln and .vcxproj files

from-file is a "best guess" approach. It's impossible to predict exactly what dlls a program will actually load. Applications can and do modify library search paths. They may also load libraries via const char* and manually call GetProcAddress.

Here's the pseudo-code for from-process-name

• search running processes for foo.exe
• get process handle from PID
• find all relevant pdbs
  • query listed of loaded modules
  • build list of module directories
  • build symbol search path
  • load pdbs via SymInitialize
  • get pdb filepath for each module
• foreach pdb
  • extract list of source files
• foreach source file
  • resolve local file path
• generate .sln and .vcxproj files

The difference is that from-process-name is doing zero guess work. It's getting the actual list of loaded modules and getting the actual pdbs for those files. The call to SymInitialize will even fetch pdbs from a custom symbol server if one is specified.

If you have a single file application then from-file is sufficient. If you have a complex application that loads a variety of modules then from-process-name is ideal. There's also from-pid if you need to be explicit.

Win32 API Reference

This project was moderately painful to get working. Figuring out the exact Win32 API calls to make was a grind. Here are the APIs calls that from-process-name needs to make:

1. OpenProcess
2. EnumProcessModules
3. GetModuleFileNameExW
4. ImageLoad
5. SymInitialize
6. SymGetModuleInfo64
7. SymCleanup

If you look at the source code there's also some deep magic to rip data out of the pdb. I'm pretty sure all the code on GitHub for this originates from PEDEUMP code 1998 by Matt Pietrek. It's awful.

Also, if you call Sym* functions your process needs to be able to load symsrv.dll. Which you may need to include in your deployment. The fts_autosln pre-built binaries includes a copy of symsrv.dll.

Separation of IDE Concerns

I wish to go on a quick tangent.

Modern IDEs are bloated kitchen sinks that perform too many functions. Each function may by filled by a variety of tools. There are three functions I care about today.

1. Build System: Visual Studio (MSBuild), Make, SCons, Buck, Bazel
2. Code Editor: Visual Studio, VS Code, Vim, Emacs, Sublime Text
3. Debugger: Visual Studio, VS Code, WinDBG, RemedyBG

Build, code, and debug. Lots of tools can do one or the other. Some tools, like Visual Studio, try to do everything.

The three functions can and should be fully orthogonal. Build systems vary by company. Code editors vary by individual. Visual Studio is still hands down the best debugger available.

My workflow at the moment is Buck/Unreal for Build Systems + 10x for Code Editor + Visual Studio for debugging. Visual Studio can easily debug any process built by any build system. All you need are pdbs and source code. Visual Studio does NOT have to be the build system. fts_autosln exists so I can generate thin-ish .sln files for extremely large and complex projects.

Future Work

Right now fts_autosln does everything I need. There's a few areas that might be interesting.

1. Better launch support. Right now Visual Studio will launch your exe when you press F5. It might be worth letting users specify things such as working dir, env vars, multiple modes, etc. My primary use case is "attach to process" so I punted on this.
2. Source indexing support. When `fts_autosln` generates an `sln` for a running process it will fetch `pdbs` from Symbol Servers. However it won't fetch source code via source indexing. That could be super cool. However the optimal implementation of this may vary by source control system.
3. Python support. My projects these days often involve a lot Python code running in an embedded interpreter. There's a lot of permutations in how Python is used. I haven't explored this at all yet.

If anyone finds this tool useful please let me know. If anyone has feature requires file them on GitHub.

Thanks for reading.