fix: support ipykernel 7, which has introduced a lock we need to disable

[maartenbreddels]

No description provided.

[Kirill888]

what does this solve? What happens when this change is not applied? Can we solve this without making even more assumptions about implementation details of the kernel we are wrapping?

[maartenbreddels]

Good question, with ipykernel 7, new messages don't even get processed while jupyter-ui-poll is in control. So in the typical example of increasing the counter on the button description, nothing happens, no event handler gets called, nothing. That is because in ipython/ipykernel#1430 this lock was introduced:

        async with asyncio_lock:
            await self.dispatch_shell(msg, subshell_id=subshell_id)

So execute_request, or any shell message, like events from the comm's get stuck due to this lock.

[ianthomas23]

If you have issues with ipykernel you should report them there and not just talk about it amongst yourselves. Better still, contribute some developer time there if your code here depends on it.

[maartenbreddels]

I’m not sure why this is framed as not taking upstream responsibility. I linked to the upstream PR deliberately to leave breadcrumbs for others hitting the same issue.

At the time, I didn’t know whether that specific upstream change was problematic, and I didn’t have time to investigate further.

Fortunately @jph00 had time to dig into it — which is exactly how open source collaboration should work.

[Kirill888]

@ianthomas23 some would see this whole repo and package as an "issue with ipykernel", or more accurately with ipython. Essentially ipython has an assumption that all messages can and should be processed in order and one at a time only, including "execute code cell" messages. This means that any kind of "wait for user input delivered as a separate kind of message inside of cell execute request message" simply not possible, ever. Not with async, not with threads, not possible, by design. It IS somewhat kinda possible when ipykernel is the kernel, and it's due to how it is implemented, but this is only possible with "almost monkeypatching" that this module does. Jupyterlite for example enforces message ordering and non-concurrency in JavaScript, making it impossible to patch in Python land at all (see #23).

So, not sure how one is supposed to communicate "hey we are relying on some internal implementation details of your code, and you changed that, and it's not working no more for us". It's not like this is a very high-use, high impact project, just an itch to scratch from ages ago, that I keep maintaining for some stupid reason, I don't even use this myself no more.

[ianthomas23]

My apologies, my previous comment was ill-considered and more aggressive than I intended.

There are a large number of projects that depend on ipykernel, and many are annoyed at some design decision or implementation detail of it. Equally, any attempt to change something in ipykernel annoys a different subset of those downstream projects. For example there has has been some effort to support concurrent async execution over the last couple of years, but those experiments have stopped and I don't see anyone willing to continue working on it. It seems impossible to simultaneously satisfy the desires of those wanting change and those wanting backward compatibility.

Anyway, what I should have written originally is that there isn't a coherent set of ipykernel maintainers who consider themselves responsible for downstream projects, there are just a few maintainers keeping the lights on and ensuring that their particular use cases continue to work. Use cases outside of those are probably going to be ignored, so anyone interested in those would probably need to turn up in the ipykernel repo and get involved there. I understand that is unlikely to happen.

[jph00]

@ianthomas23 I did try to do exactly that, FWIW. I found a downstream problem (in the oldest/original downstream project, which is still widely used), and identified a solution, and also pointed out that the current ipykernel has outstanding problems that the current tests aren't addressing. The PR happens to fix a failing test, but if fixing a test breaks a project, then the test is broken, not the project.

We certainly have both time and money to help with ipykernel, and for now are spending that on forks and alternatives, because it's not been clear how one becomes a maintainer and have ipykernel address projects beyond the particular use cases of the existing maintainers.

I'm not complaining about this situation - it's open source software and no-one should feel compelled to do anything. Just making you aware that the problem is not a lack of resources, or a lack of interest in the project. But when a project focuses on the needs of the current maintainers, it means that other projects, even if they have resources to contribute, don't have an obvious way to get involved.

If there are maintainers who would be interested in either taking on a contract to add and fix more tests to address nbclassic issues (and create a more rigorous set of tests that will benefit all downstream projects), or in bringing on new maintainers who could do so, I'd love to provide that financial support for the project.

[Kirill888]

My apologies, my previous comment was ill-considered and more aggressive than I intended.

@ianthomas23 I get it, open source development is collapsing and it is sad. Anger is one of the appropriate reactions here in my opinion.

I'm not complaining about this situation - it's open source software and no-one should feel compelled to do anything. Just making you aware that the problem is not a lack of resources, or a lack of interest in the project.

@jph00 I would disagree, I think developers should feel compelled to improve open source software they chose to be responsible for. Compelled by the sense of duty and commitment to the collaboration effort. It's very human to work collaboratively, and open source development is meant to be a social activity. It is not anymore. It's not just LLMs removing the need for a typical developer to engage with experts in the specific open source software stack. It's extreme de-funding of software development and other knowledge work that can not be framed as "pivot to AI". But most important factor in my opinion is a break down of social contract. There are no clear paths to relative financial safety through honest effort no more, even in the so called "developed" countries. Knowledge workers in particular went from relatively empowered, well compensated and in demand to "I was restructured away and nobody is hiring for my skills no more". Knowledge work market is being cornered with subsidized compute and total ban on funding anything unrelated to the current hyper-focus darling - LLM.

We certainly have both time and money to help with ipykernel, and for now are spending that on forks and alternatives

And as LLMs get better and maintainers get more and more disillusioned this will be the norm. Just loads and loads of private forks and vendored in dependencies (because security) being copied around behind closed doors by some agentic ai.

If there are maintainers who would be interested in either taking on a contract to add and fix more tests to address nbclassic issues (and create a more rigorous set of tests that will benefit all downstream projects), or in bringing on new maintainers who could do so, I'd love to provide that financial support for the project.

Having a contract here and there for a specific task is all good an well. But this does not create sustainable model for open source software development or even minimal maintenance. Contracts are transient and unpredictable, they come with huge overheads and some legal risks, they force ordering of development that might be long term negative to the project. How about a grant/stipend with a long term commitment guarantee to a trusted group of developers with minimal strings attached. I bet this would result in a way more value for money and goodwill being produced. But what CFO will be crazy enough to support that kind of investment? And it is investment not charity.

[jph00]

It sounds like you're you're in a really rough situation, and that sucks. I still maintain >100 OS projects and I'm loving it, and know lots of folks that feel the same way. I'm not sure what's causing this bifurcation, but I hope we can find ways for more OS devs to re-find the joy, and also get financial support. I find successful contracts normally lead to ongoing financial relationships, when that's something folks are interested in, although I wouldn't jump straight to an ongoing commitment, personally. Nothing crazy about that - although I'm sure it's not for everyone. (Our OS forks/changes are also OS, BTW, and I don't vendor stuff in. It's not really sustainable to do that IMO.)

…

Message ID: ***@***.***>

[Kirill888]

@jph00 bifurcation is economical, with age being the main causal factor. Ask some of your younger employees how many years they were renting before they could get a place of their own, if they managed that at all. Ask them how many after tax yearly incomes they had to promise to give up for that privilege. Ask them what proportion of their income went into the rent before that. Compare with your own experiences.

Other main factor is "user facing" vs "developer facing" projects. There are projects out there that many people use daily, but few people even know exist, as these are foundational packages that only developers of higher level, more visible packages rely on. Visibility vs impact.

I find successful contracts normally lead to ongoing financial
relationships, when that's something folks are interested in, although I
wouldn't jump straight to an ongoing commitment, personally. Nothing crazy
about that - although I'm sure it's not for everyone.

First off, right now is not "normally". Secondly, that's exactly my point. It is financially irresponsible for any single company to fund open source development without lawyers getting involved. And we do not have many reliable non-corporate sources of funding for ongoing maintenance of important open source projects like ipython. It is a structural problem made worse by significant discounting of the value of software in general in the eyes of people making financial decisions (LLMs can do it cheaper, or will soon).

I still maintain >100 OS projects and I'm loving it, and know lots of folks
that feel the same way.

Awesome, happy for you, and I'm sure users of your software enjoy it and derive value from it. But you are a well-established, well connected, clearly brilliant, entrepreneurial individual with a freaking wikipedia entry to your name. Your experience will always be different from the norm.