Extending Python with AI: semantic APIs

How AI might change the code we write

It’s an exciting time to be thinking about the art of programming, in particular because of current and upcoming advancements in AI. There are many smart people thinking about AI and programming, but this post is about an avenue that I haven’t seen pursued: putting AI features into programming itself. Let me start by explaining what I mean by that.

AI + programming

There are a number of ways that AI and programming can intersect, and I haven’t seen people talk about the differences, so I’m taking the liberty of introducing names for them to explain how this post is different.

The primary thing that I’ve seen people talk about is what I call “AI for programming”. The prototypical entry in this space is Copilot: an AI assistant for writing code. I broadly describe this space as using AI to assist with the current task of programming: keeping “programming” the same (the code looks the same in the end), but replacing human effort with AIs. There are many cool ways that people are thinking of tackling this, which I’m not going to get into.

Another thing that I’m mentioning for completeness is “AI via programming”, which is where people use a programming language to build/investigate/deploy AI models. I’m not going to talk about that here either.

The thing that I’m interested in, and haven’t seen people tackle, is “AI in programming”. Can we evolve the notion of what it means to program in the first place? Can we devise new programming paradigms that are only possible with AI, so that programs will look fundamentally different than they do currently?

I definitely haven’t cracked this, but here are a few fun + thought-provoking ideas in this space.

API mapping

One thing that I’ve always thought is primitive about modern programming is how much is based on exact string matching. If you spell a variable wrong, or capitalize it differently, or use underscores instead of capitalization, then the computer will have no idea what you’re talking about and the program won’t work. I’d say it’s generally considered a feature that natural language is more flexible than this.

One can imagine simple use cases for this: maybe a programming language allows simple typos or case differences.

I think a much more interesting use case is when consuming APIs, particularly external ones. There’s a laborious process of reading documentation and mapping one’s use case onto it, and while this is being somewhat automated via tools like Copilot, this currently only works for public APIs which are already widely used.

The next section goes into why I think this is really interesting, but here’s a somewhat-interesting example: I often remember the semantics of an API, but not the exact strings that I need to write down to invoke it. The example that comes to mind is that different HTTP libraries have different keyword names for passing request data, and I never remember which libraries call it which things. But I know they are all variants on the concept of “data” or “content”, and why can’t the computer help with the rote task of mapping that semantic concept onto the API keyword name?

I wrote a proof of concept library that offers semantically-fuzzy keyword name matching. For example:

from fuzzy_interfaces import fuzzy_keywords

@fuzzy_keywords
def myFunction(data):
    print("Was passed %r" % data)

# This works like normal:
myFunction(data="hello world")
# This also works:
myFunction(content="foo bar")

I also extended this to fuzzy matching on the name of the API function as well.

Fuzzy polymorphism

The above use case is fun to think about and I could imagine it saving myself a small amount of time, but it’s also something that Copilot helps with, and is essentially a static problem.

But the thing that I find fascinating about this approach is that because it’s done at runtime, we can apply it dynamically, and perhaps more importantly polymorphically: differently for different objects, even at the same code site.

Here’s a toy example that shows what I mean:

from fuzzy_interfaces import fuzzy_getattr

class Car:
    def drive(self):
        print("Car.drive()")

class Bike:
    def ride(self):
        print("Bike.ride()")

def driveVehicle(vehicle):
    func = fuzzy_getattr(vehicle, "drive")
    func()

driveVehicle(Car())  # prints "Car.drive()"
driveVehicle(Bike()) # prints "Bike.ride()"

Maybe we can call this “fuzzy polymorphism”: the ability to operate on objects that are “similar” but don’t offer exactly the same api.

I’m not actually proposing this particular approach, and I’m not sure that attribute and keyword names are actually the right place to derive semantics. But my point is that by adding semantic reasoning to a language we can think about having fundamentally new models of coding. We’ve had “interfaces” for many years, that originally started as binary interfaces of exactly what data had to be put in what registers, and currently we have textual interfaces that are about string function names and argument names. But now we can start to think about “semantic interfaces”, where invocation and provision are at the semantic level, not the textual level.

API bridging

As I mentioned, I don’t think simply remapping function and keyword names is going to get us very far — practically-speaking, even very-similar APIs will differ a bit more than that. This is what I’m actively experimenting with.

I don’t have anything concrete yet, but we can get a sense of what this might look like by bringing in a modern LLM. I personally don’t think this is the correct long-term technique for this particular problem (because inaccuracies are unacceptable), but that’s a point for another blog post.

Here’s an example of something that one might call “API bridging”: we have a particular semantic interface that we want, and we have one or many objects that are capable of providing the relevant functionality, but don’t have a function that is exactly that semantic. In this particular example, lets say that we want to GET a url with some request data, and we want to receive the status code and response text. Further, we want to be able to do this any of the HTTP libraries in Python.

The exact scenario isn’t super important, but the point is that the desired functionality is available, but there isn’t a single function of the exact shape that we want. So let’s ask an LLM to synthesize the bridging code:

def getWithLibrary(url, request_data, library_to_use):
    function_text = my_favorite_llm.query("""
        Please write a function that, using the %s library,
        takes a url and a dict of request data and GETs the url
        and returns the status code and response text as a tuple
        """ % library_to_use.__name__)

    function = eval(function_text)

    status_code, response_data = function(url, request_data)
    assert status_code == 200
    return response_data

When I run this by hand, ChatGPT is able to synthesize functions that use the actual APIs and wrap them in a function of the specified shape. This is a pretty simple example, but it shows that it’s possible to live in a world where AI does much of the work of bridging between a requested set of semantics and the exact shape of an API that we are trying to use.

Future direction

We’re starting to get into the world of “program synthesis”, which could also be a fundamental shift in how we think about writing software. But personally I view this as “here be dragons” territory that, while intellectually fascinating, is too nascent to be the basis of the next set of language features.

I’m currently experimenting with what the next killer language feature could be, but in a more-prosaic “what next level of abstraction is just becoming available” fashion. AI is certainly one place that I’m looking, but there’s also the relentless march of “compute time gets cheaper over time while developer time gets more expensive over time”.

If you’re interested in these sorts of things hit me up on Twitter!