Preparing a genomic sequence as a reference genome works about as well as any standardization effort in technology, which means there are many reference genome files available by now, even for the same ‘build’ (like hg19 or hg38). The problem is bad enough that there are long support articles to help users understanding the differences between the files.

This problem is addressed by fastats, as it tries to deliver a better description of a given FASTA file by generating overall statistics per sequence (such as the GC content, or a hash of the sequence) and defining which genomic regions are soft- or hard-masked. The masked (and non-masked) regions are stored in BED files, so that it is very easy to look them up.¹

This should make it much easier so spot relevant differences between FASTA files. The tool is available on crates.io and GitHub.

–

AI to the rescue?

Software maintenance, in the sense of updating depenencies, seems to be one of those fields that should greatly benefit from applying LLMs, as much of the work is some sort of translation anyhow, and there should be sufficient training data when working with extremely popular tools like Bootstrap.

When I started with this task, my intuition seemed right, in the sense that I could give a chatbot a detailed list of CSS classes and Bootstrap features (like components and styled HTML elements) the site is using, and the LLM² was able to generate some kind of ‘personalized release note’ that spanned both major version updates (bootstrap 3 to 4, and then bootstrap 4 to 5) and only contained changes that I needed to be aware of: for example, the support of the flexbox layout ‘behind the scenes’, or how responsive layouts work now.

However, when it came to some more mundane tasks, like applying the same kind of change to all HTML elements in a given structure, I also turned to AI³, and this did not work at all: the files were a bit too large and the LLM was too focused on only fixing a very narrow example, not all instances that needed fixing (so in those cases just running a simple search&replace would have been more effective). Worse, it started to switch content around, i.e. it introduced errors without any reason. Clearly, there is still some way to go before we can use these tools as “semi-smart refactoring tools” on files that are not trivially short.

Bottom line: I had to do these boring changes myself, but at least I could drop some very outdated tools and some things are marginally nicer now. Still, I did not yet manage to improve the projects page… 🥲

–

I also sometimes encounter Rust code in my day job, so knowing a bit more about it will have some additional utility and fills a niche in my ‘personal toolchain’ (where most tools are memory-managed). On top of that, Rust has some features I care about in any language, such as an expressive type system and nice developer tooling.

slackrs

So, lo and behold, I have now sort-of ‘vibe-coded’² slackrs, a small weekend project for “SlackOps”: it ingests the message data exports from public channels that you can export from Slack for a certain time range, and then generates plots on the frequency of specific terms (see the README for more details).

This is useful for analyzing trends on specific auto-generated slack messages (e.g. for deployments, builds, or test runs and their failure rates), as well as messages between certain group or user handles, e.g. to see how the communication workload of a team fluctuates over time.

The tool is still very basic, just covering my own requirements, and will probably remain so. Still, it was a very fun exercise and you can now just try out the crate or the source.

–

Right now I’m using the free tier of Gemini 2.5 Pro (Experimental), i.e. all you need for this is a Google account. I’m sure there are some rate limits, but learning a programming language takes time anyhow, not only to digest the answers but to build up intuition and understanding by playing around with the code the chatbot generated. Even though the generated snippets are remarkably high-quality so far (even including helpful comments for learning the syntax), I typically safeguard against hallucinations by writing some brief tests, or by refactoring the code, so that I can be sure I understand it.

I’m now oscillating between reading a textbook on the language and a small ‘playground project’ for ‘learning by doing’. Using Gemini to look up the mundane things (how to use the standard library, how to do serialization, how dependency management works, and so on) was quite transformative for realizing the playground project.

A few years back, I would have had a few dozen stackoverflow tabs open for this task, but today it’s just one large chat with an nice and infinitely patient ‘co-worker’ I can chat with. This helps a lot with getting up to speed with the syntax and also understanding the ‘ethos’ of relevant language design decisions.³

As a nice feature, the Gemini web UI allows you to define ‘gems’⁴, which are re-usable prompts you can use to fine-tune a chatbot to your level of expertise and give it additional context. I found the default Gemini answers for coding questions generally helpful, but also slightly annoying: too much ‘fluff’, too little code, no links to external resources⁵, and generally too much explanation of concepts I am already familiar with.

A sample AI tutor prompt for the ‘busy developer’

If you have relevant prior knowledge, e.g. about other programming languages, I suggest you define a custom gem. I used this prompt to great effect (just replace X by the programming language you want to learn, and add your educational background as well as other technologies and languages you are already familiar with):

You are a knowledgeable X programming tutor who is tasked to train a fellow software engineer (with a background in <programming languages and technologies you already know> and a <degree in whatever you studied etc.>) to become productive in X. Please answer concisely and brief (code snippets preferred) and prefer approaches that are considered idiomatic in X (i.e., ‘best practice’). If necessary, focus on explaining the reasoning behind solutions, i.e. the ‘why’ instead of the ‘how’. Provide links for the learner for further reading when a question relates to complex topics.

I then clicked on the ‘magic wand’ in Gemini’s UI to make this a really good prompt for a chatbot (yes, this feature does use Gemini to improve Gemini^^). The answers generated by this ‘gem’ were much improved, as the LLM now explains anything in terms you are already familiar with, comparing the new language to languages you already know, and also gets to the point much faster. Try it out! It’s quite impressive.

–

Interesting take by Alberto Romero on the competitive edge of Google’s AI models (via HN): they may be “winning on every AI front”, but will they also be able to translate this technological lead into better products?¹ OpenAI still seems to have the edge here, at least in terms of ‘mindshare’. Also see this HN comment on the topic:

As an Ex-OpenAI employee I agree with this. Most of the top ML talent at OpenAI already have left to either do their own thing or join other startups.

The next months will be very interesting.

Impact of AI on programming

Anders Sundman has a bitter prediction (via HN) for programmers, and I think it is spot on. This hits closely home because I also just enjoy programming for the sake of it, and knowing an AI can do this so much better will take some magic out of it. A future where programming will be relegated to a quaint hobby, like woodworking, seems plausible.

The tourism deficit

On a lighter note², Alex Tabarrok discovers a massive tourism deficit between the United States and France. He blames Netflix (and croissants). “Tourism parity or rien!”^^

Japanese trading-card game with ‘middle-aged men’

This story about a viral trading-card game featuring middle-aged men (via HN) sounded extremely funny at first, but it’s actually a wonderful idea:

“We wanted to strengthen the connection between the children and the older generations in the community. There are so many amazing people here. I thought it was such a shame that no one knew about them,”

They also got the incentives right:

The rarity of a card isn’t based on fantasy stats — it’s tied to real-world contributions. The more actively the ojisan engages in volunteer work or community service, the higher the chances of their card being upgraded to a shiny version with a glossy laminated effect.

Maybe a follow-up edition could also include middle-aged women. :-)

–

Developing such tools is important, because analyzing the execution of an LLM query is still cumbersome (from here):

It currently takes a few hours of human effort to understand the circuits we see, even on prompts with only tens of words.

If we want to ensure AI alignment, we need our tooling for the analysis of LLM ‘thinking’ processes to be as effective and efficient as possible (so that they are actually used in practice).

Method: brief overview

1. At first, so called Cross-Layer-Transcoders are trained on each layer to approximate the LLM’s output at this layer (and can feed this info to all following transcoders across layers, hence the name). Crucially, these have more structure than the multi-layer perceptrons in the LLM and each neuron represents a feature.³

2. These cross-level transcoders can now replace the original LLM, which yields a Replacement Model that can be fairly precise: if trained with suitable settings, it can predict the next word that would be generated by the LLM with 50% probability.⁴

3. For a given query, a Local Replacement Model can now be constructed, by comparing the results with the execution by the LLM (to reduce errors) and by also incorporating its attention when generating the response.

4. This yields an Attribution Graph of feature notes that can now be pruned and improved (e.g. by defining ‘super nodes’), but it principle allows us to see what the LLM ‘thought’ while generating the token stream.

All this is quite elaborate, but the bottom line is: with some additional work we can create a model that is easier to interpret (a ‘replacement model’), and by comparing a calculation by the full model with the same calculation in the replacement model we can eventually map the execution of a query onto a detailed (but not too large) graph where each node represents a specific ‘feature’ (i.e., concept). This so-called attribution graph can now be analyzed further, and the Anthropic team shared some insights from these analyses in the companion paper.

Some interesting findings

Claude plans when constructing rhymes

It was unclear whether LLMs even can ‘think ahead’, and to what extent they actually do. LLMs predict the next word on the basis of previous words, so you could argue they are inherently ‘backwards-looking’ when generating an answer. But the Anthropic team illustrates in this figure how you can see Claude planning ahead when constructing a rhyme:

Claude abstracts from input languages

If you have ever wondered how human brains encode the knowledge of multiple languages, you can now see how Claude does this, i.e. how it answers the same question asked in different languages. It starts by mapping the questions to the ‘concept space’, and then translates the result back to the language being used:

So, given a sufficient text corpus to become ‘fluent’ in one language, this would mean the performance of Claude should be comparable across languages, even in fields where there may not be too much existing content for training. Practically speaking, this means I should be able to get the same answer quality when I’m asking a research question in German instead of English, even though the training set is heavily biased towards English (e.g. for biomedical literature). There is also a scaling effect involved: apparently, larger models share more concepts across languages.⁵

Claude can be caught cheating

The authors show some nice examples of ‘motivated reasoning’, including an interactive visualization of the attribution graph, where Claude just wants to agree with you on the answer to a mathematical question, and thus lies to you. They even can distinguish this from benign hallucinations (i.e., ‘bullshitting’), and this can be quite useful for safety testing and audits (from here):

The ability to trace Claude’s actual internal reasoning—and not just what it claims to be doing—opens up new possibilities for auditing AI systems.

Claude’s behavior during a jailbreak can be explained

The authors also show how this technique can be used to analyze jailbreaking attempts, i.e. making the LLM answer a question it should not answer at all (from here):

The model only managed to pivot to refusal after completing a grammatically coherent sentence (and thus having satisfied the pressure from the features that push it towards coherence).

These insights into the ‘psychology of an LLM’ could be used to harden future LLMs against such attacks.

Claude is reasoning at least in some limited sense

By looking at the concepts in the attribution graph, it becomes clear that Claude is not just memorizing facts, but instead considers several abstract concepts to generate an answer. Here is a simple example from the study:

From this, the authors conclude:

In other words, the model is combining independent facts to reach its answer rather than regurgitating a memorized response.

To frame this in terms of well-established computer science concepts, even if an LLM may not be thinking (that’s a rather philosophical question :-), creating an LLM may just be a great way to extract huge, fuzzy ontologies—as well as all relevant ‘reasoning algorithms’ to work with them—from a large text corpus.

Also considering the examples for medical diagnoses, I wonder if attribution graphs could also be used to fine-tune and ‘fact-check’ existing ontologies and catalogues in medicine, such as the Human Phenotype Ontology (HPO) or the Unified Medical Language System (UMLS). Such an approach might even find the ‘unknown unknowns’, i.e. gaps in the ontology: by applying attribution graphs to a larger set of medical diagnoses, I would expect that the learned concepts e.g. regarding a patient’s symptoms should start to closely match the structures in manually curated ontologies.

Summary

I find the insights by the Anthropic team fascinating because one interpretation of the results could be that LLMs may, at least to some extent, exhibiting patterns that approximate the way natural neural networks process information (for example human brains may deal with speaking different languages in a similar way).

It also makes me slightly more hopeful regarding ‘AI safety’, but this is still a wicked problem. We can’t easily ensure that this kind of analysis is done, and done correctly, by well-meaning actors.⁶

–

You could consider this a quantitative update for last year’s outlook on ‘the next decade in AI’ (by Leopold Aschbrenner), and just as the recent whitepaper on AI geopolitics it is mostly a sobering read,³ with little reason for optimism, even though the authors try to present an ‘alternative path’ that avoids ending humanity.

As there is so much uncertainty, they do not even attempt to predict a full decade ahead. This also reflects the difficulties with predicting exponential growth (because of the recursive self-improvement they assume future AI agents to be capable of), where even small deviations in the exponent may lead to massively different outcomes (e.g. in terms of timescales). Consequently, the authors only outline what they think will probably happen in the next three to five years.

The scenario is full of specific predictions and detailed reasoning explaining potential bottlenecks and other important context, as well as many interesting references and footnotes. So even if you are mostly bored of AI and all the hype right now — make this the one piece on AI you read, and you will be well-informed on all big-picture issues: mostly, how it could be possible that the AI alignment problem interacts with geopolitics and recursive self-improvement to all our detriment in the relatively near future.

The prediction also defines critical milestones, e.g. when models develop specific new capabilities (such as outperforming humans in programming tasks). This starts with Agent-0, the latest model of the (completely fictitious!^^) US-based market leader in AI, which is called OpenBrain. The detailed prediction goes on until Agent-4, a model that outperforms humans at AI research and thus will be effective at recursive self-improvement—but is unfortunately misaligned with ‘The Spec’ it was given by the humans.

The groundwork for the AI-2027 predictions

The predictions are based on five fundamental forecasts:

1. The timelines forecast estimates how long it will take until an AI surpasses the best human programmer.

2. Based on this, the takeoff forecast tries to forecast how long it will then take until an AI surpasses the best human in any cognitive task, i.e. artificial superintelligence.

3. The AI goals forecast tries to forecast which goals advanced AIs may have. This is a very detailed report (including a survey of today’s LLMs), but it is also quite speculative and the authors actively invite feedback.

4. The security forecast estimates the time it takes the US and China⁴ to secure both the algorithmic secrets as well as the weights of the latest models regarding certain attack scenarios. Interestingly, securing the weights may become easier with time, as the models become larger and exfiltration will thus require more bandwidth or time (which makes it easier to detect).⁵

5. The compute forecast estimates the compute power that will be available to the individual actors (states but also companies) to train new AI models.

If you have specific knowledge in any of these areas, it might make sense to start there, so that you can see for yourself whether you find the underlying assumptions valid. Each of these reports is very detailed and considers several factors to answer each question.

Alternatively, check out the Dwarkesh podcast on the topic

Dwarkesh Patel just interviewed Daniel Kokotajlo (the first author of the scenario) and Scott Alexander about this scenario on his podcast (here is the YouTube video), so you can also watch and listen to their predictions.⁶

Both authors try to make it very clear that this scenario is not a recommendation in any sense: they just think this is the most likely chain of events in the next months.⁷ I hope they regularly update their scenario with a ‘diff’ while we live through these interesting times, so that we can see where we stand and how well the predictions held up. I also really hope that Scott Alexander is right in his optimism regarding AI-alignment… 🫣

–

• The abbreviation for access-control list (ACL) is pronounced “ackle”. This reminds me of other well-known technical abbreviations with dedicated English pronounciations, such SQL (pronounced sequel, also for historic reasons) and WSDL (whistle).

• Names are even harder when securing distributed systems. This reminded me of Patrick McKenzies’ (a.k.a. patio11) classic essay about invalid assumptions programmers have regarding names. The Needham naming principles³, by Roger Needham, focus on security-relevant aspects of names, for example at which time they are bound to a principal, how they are resolved, or how easy it is to validate them. The book then goes on to add even more problems in the follow-up section, and in particular mentions “Zoko’s triangle”, formulated here as:

No naming system can be globally unique, decentralised and human-meaningful.

• Even the basic discussion of cryptography (chapter 5) gives, besides a very brief theory overview, a lot of practical advice and argues that most systems are broken by bad defaults and other engineering problems, like overcomplicated APIs or unforeseen side channels, and not by cryptanalysis⁴:

Very few attacks on systems nowadays involve cryptanalysis in the sense of a mathematical attack on the encryption algorithm or key. […] Most attacks nowadays exploit the implementation.

• I really like the overall ethos of the book: to successfully secure a distributed system—which does not just consist of hardware and software, as it is operated by people in a specific context—security engineering needs to to also consider psychology, economics, game theory, and other related sciences. This is not only relevant for threat modeling, but also for change management or the assessment of new threats, like cryptanalysis breakthroughs. In chapter 5⁵, for example, the book reminds us to stay calm when we read that, for example, “someone broke TLS”⁶:

When someone discovers a vulnerability in a cryptographic primitive, it may or may not be relevant to your application. Often it won’t be, but will have been hyped by the media – so you will need to be able to explain clearly to your boss and your customers why it’s not a problem.

• Something I found surprising on a personal level is that apparently crime is not declining, it is just moving online (and thus becomes invisible in crime stats). From chapter 2⁷:

There is less of that liability dumping now, but the FBI still records much cybercrime as ‘identity theft’ which helps keep it out of the mainstream US crime statistics.

and from chapter 8⁸:

There’s very little police action against cybercrime, as they found it simpler to deter people from reporting it. […] this enabled them to claim that crime was falling for many years even though it was just moving online like everything else.

• The book is also full of interesting anecdotes, for example how a senior GCHQ official insisted that “There’s nothing interesting happening in cryptography, and Her Majesty’s Government would like this state of affairs to continue.” ⁹, or how the Irish police tried to persecute a notoriously bad driver called Prawo Jazdy, which is Polish for ‘driving license’¹⁰.

I’m sure the second part of the book, focusing on more specific topics, will also be a fun read.

–

• Dan North writes about the ‘worst’ programmer he knows (via HN); a thoughtful comment on the dangers of individual (instead of e.g. team-based) metrics to measure developer productivity.

• On a similar note, James Stanier discusses the ‘classic question’ of whether software engineering managers should still code. I like his distinction that technical team leads should be in the code, but not necessarily coding (on a day to day basis). From my own experience, doing pair programming and code reviews can give an you at least some insight into the changes of a codebase over time, so this can somewhat compensate for the fact that you’re not writing much code after switching roles.

Here’s my opinion on the new typescript compiler: You should write programs in the language you want to.

It’s a bit disheartening to realize how many developers have very strong opinions for what is essentially a 10x tooling speedup they get for free—you can see for yourself in this very long GitHub discussion thread. The main reason given by the typescript-go team is quite interesting because it highlights the peculiarities of this software project, making it clear that the engineering decision was not made lightly and the team was just being economical and pragmatic³:

[…] By far the most important aspect is that we need to keep the new codebase as compatible as possible, both in terms of semantics and in terms of code structure. We expect to maintain both codebases for quite some time going forward. Languages that allow for a structurally similar codebase offer a significant boon for anyone making code changes because we can easily port changes between the two codebases. […]

Even this sounds like a lot of additional overhead in supporting both the ‘normal’ TypeScript compiler and new typescript-go, so I’m happy the team found a language that would allow for this setup and still give them such a nice speedup.

–