Add time/space benchmarks

[asolove]

⚠️ This is work-in-progress, not ready to merge.

I started writing simple FRB benchmarks. This one seems to show that we're leaking an array. The memory leaked works out consistently to a not-coincidental ~32bytes per item in the input array.

What should sensible output from these be? Actual numbers, or perhaps a limit on space/time and then each test either passes or fails?

Here's the current, very verbose, output:

FRB {
    "before": {
        "nodes": 23165,
        "time": "2013-10-15T15:35:18.000Z",
        "size_bytes": 3215944,
        "size": "3.07 mb"
    },
    "after": {
        "nodes": 24073,
        "time": "2013-10-15T15:35:21.000Z",
        "size_bytes": 3628504,
        "size": "3.46 mb"
    },
    "change": {
        "size_bytes": 412560,
        "size": "402.89 kb",
        "freed_nodes": 2726,
        "allocated_nodes": 3634,
        "details": [
            {
                "what": "Array",
                "size_bytes": 318400,
                "size": "310.94 kb",
                "+": 1588,
                "-": 1186
            },
            {
                "what": "Code",
                "size_bytes": 77104,
                "size": "75.3 kb",
                "+": 752,
                "-": 751
            },
            {
                "what": "Dict",
                "size_bytes": 192,
                "size": "192 bytes",
                "+": 4,
                "-": 0
            },
            {
                "what": "Number",
                "size_bytes": 80,
                "size": "80 bytes",
                "+": 5,
                "-": 0
            },
            {
                "what": "Object",
                "size_bytes": 1840,
                "size": "1.8 kb",
                "+": 46,
                "-": 0
            },
            {
                "what": "RegExp",
                "size_bytes": 792,
                "size": "792 bytes",
                "+": 11,
                "-": 0
            },
            {
                "what": "Scope",
                "size_bytes": 128,
                "size": "128 bytes",
                "+": 2,
                "-": 0
            },
            {
                "what": "String",
                "size_bytes": 13272,
                "size": "12.96 kb",
                "+": 619,
                "-": 195
            }
        ]
    },
    "time": 3366
}
Naive {
    "before": {
        "nodes": 22375,
        "time": "2013-10-15T15:35:21.000Z",
        "size_bytes": 3605136,
        "size": "3.44 mb"
    },
    "after": {
        "nodes": 22260,
        "time": "2013-10-15T15:35:21.000Z",
        "size_bytes": 3597840,
        "size": "3.43 mb"
    },
    "change": {
        "size_bytes": -7296,
        "size": "-7.13 kb",
        "freed_nodes": 182,
        "allocated_nodes": 69,
        "details": [
            {
                "what": "Array",
                "size_bytes": 3840,
                "size": "3.75 kb",
                "+": 34,
                "-": 51
            },
            {
                "what": "Code",
                "size_bytes": -6464,
                "size": "-6.31 kb",
                "+": 17,
                "-": 34
            },
            {
                "what": "String",
                "size_bytes": -896,
                "size": "-896 bytes",
                "+": 2,
                "-": 22
            }
        ]
    },
    "time": 130
}

[Stuk]

Is something that can be loaded into the inspector Profiles tab feasible?

[asolove]

I was hoping to get programatic access to the retain paths but I can't find one, so perhaps I'll migrate this to webkit-devtools-agent.

[kriskowal]

Perhaps CSV. This is all for naught without a way to visualize the graphs, and loading it up in a spreadsheet seems like a fast-path, and other tools wouldn’t mind CSV either generally.

[kriskowal]

And it would probably be good to set some of this up for pass/fail so we can verify that we’ve stomped out the leakage.

[kriskowal]

I think we’ll need to adapt this case to measure churn in addition to leakage to evaluate the removal of once #20 and the removal of autoCancelPrevious for the anticipated churn improvement in addition to verifying that it does not introduce leakage.

[asolove]

I'm looking at this and I think there are basically three things worth doing:

1. For now, just measure and get numbers for churn and leakage for various interesting use cases.
2. When we know the biggest problems, an automated way to get heap snapshots before and after a specific use case so we can look at the retain graph and find the problem.
3. Once we like where the numbers are, automated tests that fail if things change drastically.

[hthetiot]

@asolove could you rebase on master. Thx.