Beware of ChatGPT dazzle

I’ve long known about the power of computer visualisation to dazzle and distract, to give cover to bad or poorly supported ideas. The case study I saw at university was about a famous computer animation of the single-bullet theory in the JFK assassination.

This general idea of dazzle pops up in different contexts over the years. Most recently we can see ChatGPT and other AI tools being used in some very low-effort dazzle attempts.

Here’s a recent example in a post from Twitter:

It’s extremely difficult for two high-velocity objects on perpendicular flight paths and different altitudes to intersect with each other. It’s just as nearly impossible whether deliberate or accidental.

There is only one scenario where the probability is near certain, which is if it’s calculated i.e. engineered.

Below is the math to calculate such a perpendicular interception vector… so imagine the chances of it happening accidentally…

(See this gist for the code.)

There are so many things wrong with this tweet, I’m not going to pick over everything, but it’s fractally wrong.

The main gambit of the tweet can be paraphrased as:

this code looks complicated so the chance of the thing I am modelling happening by chance is very low

Hopefully it barely needs saying, but: there is absolutely no link between code complexity and likelihood of some probability in its problem domain being high or low.1

And then we get onto the actual code. To me, it looks like ChatGPT output, given the overly-helpful explanatory comments littering everything.2 It’s a python function, but there’s no usage examples provided.

Notice that the code is doing things in 2D, not 3D. This effectively removes the “different altitudes” part from the original tweet, and guarantees that these orthogonal flight lines3 will intersect at some point. In 3D you could have plenty of orthogonal flight lines that don’t intersect anywhere (due to the altitude aspect); what I’m saying is that using a 2D example is a weird way for the tweet author to try to make their point. They’re straw-manning their own argument.

If you try a few invocations to the function given, it doesn’t take long to find out it’s broken:

# works: outputs ((0.0, -1.0), 1.0)
print(perpendicular_interceptor((-1, 0), (1, 0), (0, 1), 1))

# doesn't work: outputs (None, None), because we've mixed floats (1.1) and ints (1 etc)
print(perpendicular_interceptor((-1.1, 0), (1, 0), (0, 1), 1))

# doesn't work: outputs (None, None) because I changed velocity from 1.0 to 1.1
print(perpendicular_interceptor((-10.0, 0.0), (10.0, 0.0), (0.0, 1.0), 1.1))

# doesn't work: outputs (None, None) because I changed start pos from (0.0, 1.0) to (0.0, -1.0).
# This should give an intersection because the only sensible interpretation of scalar
# velocity is that it is heading towards the potential intersection
print(perpendicular_interceptor((-10.0, 0.0), (10.0, 0.0), (0.0, -1.0), 1.0))

It’s very possible that I am the first person to run this code!

The code is doing too much for the simple problem it’s solving. And this part is a massive no-no:

    if abs(dvx) > 1e-12:
        tx = dx / dvx
        t_candidates.append(tx)
    
    # From y-component if dvy != 0
    if abs(dvy) > 1e-12:
        ty = dy / dvy
        t_candidates.append(ty)

Why? Because for the vast majority of problems you don’t need to split vectors down to their individual components. Vector maths and geometry can be extremely elegant (and pleasing)! If you see repeated code working on different vector elements it’s a potential code smell.4

Weird, verbose and around-the-houses techniques like this code are more likely to result in a faulty implementation.

It’s worth stressing that humans, and AI, can produce many working solutions to a given problem – and a lot of those solutions will be sub-optimal.

What’s an elegant solution to this?

The python code above is solving (or attempting to) a particular case of a general problem: intersection of two moving points in linear motion. It limits the problem to the points traveling at right angles to each other (orthogonally), and is framed in 2 dimensions. It also seems to gloss over the fact that the objects will have some sensible size – it just uses a very small number for the proximity detection.

For realism, we want to move to the 3 dimensional problem5, and solve the general problem where two points can be traveling in arbitrary directions.

I believe the key question the algorithm needs to answer is:

  • What is the minimum distance between the points during their movement?

Once you have this part cracked, you can think about how set limits on that minimum distance to decide if a collision has occurred.

More in a later post

  1. the trivial code examples of return 0 and return 1 neatly confirm this ↩︎

  2. the current crop of AI is trained on sites like Stack Overflow. But with SO in decline and more and more AI code popping up, does AI ultimately end up consuming its own earlier output? Including of course bad AI code ↩︎

  3. by ‘flight line’ I mean the infinitely long line describing a moving particle’s path. A flight line goes from infinitely far in the past to infinitely far in the future. More formally I think you’d call this the trajectory ↩︎

  4. how would you extend this code to work in 3d? Add another if abs bit? Well written vector code scales easily, this code doesn’t ↩︎

  5. a good vector solution would handle any amount of dimensions ↩︎