Show HN: Unref – A Python dead code finder built on Ruff's parser

github.com

57 points by McAvey 2 days ago

I wanted a good Python dead code finder that just works, with minimal config, until Ruff adds support for that (https://github.com/astral-sh/ruff/issues/872).

None of the newer alternatives to Vulture worked out of the box on my codebases, so I decided to build one.

The first version was in Python, then I switched to Rust for speed and to reuse Ruff's parser. It checks all of CPython in under a second.

It found lots of interesting dead code in my team's codebases, and it's the first time in years we've had a Python dead code finder in our CI.

Feedback is very welcome, especially dead code it misses.

peterabbitcook 13 minutes ago

Did you compare what you’ve built to some of the internals of nuitka? nuitka stops dead code search at the module level, but their exclusion/inclusion flags/rules might steer you towards fewer false positives for common packages.

selcuka 9 hours ago

Cool project. I think false positives are more important than the dead code it misses at this stage. For example, it marks all Django settings as unused variables, probably because settings are typically not imported directly in Django projects. For example:

    from django.conf import settings
    ...
    if settings.DEBUG:
        ...

Currently unref thinks that DEBUG is not used.

Similarly, the `application` variable in asgi.py (or wsgi.py) is passed as an argument to the ASGI/WSGI server executable, but that is also marked as unused.

  • McAvey 8 hours ago

    Thanks, one of my goals was to try and make popular frameworks work well with this tool, and I didn't add Django yet.

    I'll make a PR to add some basic Django support which should address both of your points.

  • drcongo 7 hours ago

    I got a lot of false positives in a Django project too, but trying it on some non-framework tools I've built worked pretty well. Will keep an eye on this, thanks.

  • pennomi 3 hours ago

    I wish they made Django a little less “magic”. Virtually every harness has to create workarounds specifically for Django, and simple things like settings probably don’t need to break standard coding practices.

athoscouto 5 hours ago

I've been trying to do something like this internally too. Also started with Python, then moved onto ruff. My goal is a bit broader though. I want to use the dependency graph to detect changed parts of the project to selectively trigger image builds and run tests.

Results are still poor, specially on the tests side. So far the main learning has been that a tidy codebase (dep graph) is essential for it to work properly.

I'm surprised I didn't find more open source tooling tackling this effectively.

  • McAvey 2 hours ago

    Yeah and I think these problems are only increasing in importance in AI times, with code generation being so easy.

tomtom1337 5 hours ago

What is the definition of dead code that Unref uses? I distinguish between apps and libraries. In apps I can see how any code not callable from the entrypoint is dead, and for libraries I guess a definition is any code that is not callable from a public api?

Would be nice to see the definition used placed in the docs/github readme.

  • McAvey 2 hours ago

    Yeah exactly, for apps it's any code that is not reachable from the app entry point, and for libraries is any code not reachable through the public API.

    I've updated the README and added a FAQ section in the docs to explain this, thanks for the question!

    Here's the PR - https://github.com/Apakottur/unref/pull/121

zeitlupe 5 hours ago

Thanks! I just tried it, it is definitely incredibly fast. Found a couple of true positives, too.

  • McAvey 2 hours ago

    Glad to hear!

    Feel free to open an issue on github if you find any false positives/negatives.

neuroticnews25 7 hours ago

It listed 204 unused functions in my flask project, most or all of them are false positives, and I don't do anything magical.

  • McAvey 7 hours ago

    Thanks for trying it and for the feedback! I'm working on making popular frameworks work out of the box, and Flask clearly needs more work.

    Feel free to open a GitHub issue with a few of the false positives: https://github.com/Apakottur/unref/issues

LaurensBER 9 hours ago

Thanks, this seems like a very good approach and is indeed very fast! Happy to run this over our codebase and see how it compares against Vulture.

  • McAvey 8 hours ago

    Great let me know how it goes!

    The initial version was in Python which was pretty fast as well, decided to try Rust both for speed and to reuse more constructs from ruff/ty for better compatibility with them.

leobuskin 10 hours ago

All of CPython’s stdlib, right? It’s Python dead code checker, not C.

  • McAvey 8 hours ago

    Yes, only the Python code (every .py file in the CPython repo).

    The benchmark script is in the repo so you can run it yourself, the results are from my potato computer, will probably be even faster on modern PCs.

    I've updated the README to be more clear about that, thanks for pointing that out!