# Bytecode Analyzer

There are two variants: automatic analysis (for a fast impression) and manual analysis (recommended)

## Automatic Analysis

In the `samples` directory, add your original and reproduction cache file.
They should come directly after another when the directory listing is alphabetically sorted (using Python's `sorted()`).
You can achieve this by, for example, naming the original file `mypackage-somefile.cpython-312.pyc` and the reproduction attempt `mypackage-somefile-3.12.3.pyc`.
Two samples from MIT-licensed packages ([table-synthesis](https://pypi.org/project/table-synthesis/) and [gator-core](https://pypi.org/project/table-synthesis/)) are exemplary included.

To print some automatically extracted information, just execute

```
python analyzer.py
```

## Manual Analysis

Reproducing byte code depends on the correct Python version. 
We recommend running a specific Python version through containers, e.g.
```
docker run --rm -it -v $PWD:/cache-files python:3.11.13-slim bash
```

You can then start a Python REPL environment with `python` and set up an ad-hoc tooling by copy-pasting the following snippet:

```py
import dis, marshal, os, sys, py_compile
from pprint import pprint as orig_pprint
def pprint(*args):
    orig_pprint(*args)
    return args

offset = 16 if sys.version_info >= (3, 7, 0) else 12  # .pyc header size changed at some point
consts = lambda fn: pprint(marshal.loads(open(fn, "rb").read()[offset:]).co_consts)
names = lambda fn: pprint(marshal.loads(open(fn, "rb").read()[offset:]).co_names)
code = lambda fn: dis.dis(marshal.loads(open(fn, "rb").read()[offset:]).co_code)
consts2 = lambda fn, accessor: pprint(accessor(marshal.loads(open(fn, "rb").read()[offset:])).co_consts)
code2 = lambda fn, accessor: dis.dis(accessor(marshal.loads(open(fn, "rb").read()[offset:])).co_code)

def find_distinct(fn, fn2, isloaded=False):
    if not isloaded:
        root = marshal.loads(open(fn, "rb").read()[offset:])
        root2 = marshal.loads(open(fn2, "rb").read()[offset:])
    else:
        root, root2 = fn, fn2
    def cmp_recursive(obj1, obj2, acc="x"):
        if obj1.co_code != obj2.co_code:
            print("Found code difference at %s" % acc)
            return
        if len(obj1.co_consts) != len(obj2.co_consts):
            print("Found constants difference at %s" % acc)
            return
        i = 0
        for c1, c2 in zip(obj1.co_consts, obj2.co_consts):
            h1 = hasattr(c1, "co_consts")
            h2 = hasattr(c2, "co_consts")
            if h1 == h2 == True:
                cmp_recursive(c1, c2, acc + ".co_consts[%s]" % i)
            if h1 != h2:
                print("Found constants difference at %s (index %s)" % (acc, i))
            i += 1
    cmp_recursive(root, root2)
```

You can then compile the bytecode with the following command:

```py
py_compile.compile("/path/to/source.py", "/path/to/repro.pyc")
```

Then, a basic manual analysis workflow looks like this:

```py
orig = "/path/to/__pycache__/source.cpython-311.pyc"
repro = "/path/to/repro.pyc"

consts(orig)  # print all consts in the original cache
code(repro)   # print the decompiled bytecode in the reproduction cache 

find_distinct(orig, repro)  # Recursively iterate and output the first difference found
                            # If nothing is returned, the reproduction is perfect (disregarding metadata which is not execution specific)

consts2(repro, lambda x: x.co_consts[1])  # consts2 and code2 is helpful for investigating nested code objects
```