Recent searches
Search options
I hate Python so much right now 
The blasted CPython implementation is shit at handling recursion, so I had to turn a nice, clean, functional one-line command into a 6-line procedural function with a while.
I hate that Python is so bad at implementing functional paradigms. Even JavaScript is better! And it's even more object-oriented!
How big is the dataset you're trying to flatten? The default recursion limit is 1000, but you can easily change that, by orders of magnitude if you like.
@cazabon It’s a chain of around 600 comments that triggers the exception. The issue is that each object has multiple fields that are “dict-ified”, and even though only one is recursive (the replies), all those calls stack up because the top function is not closed yet.
My machine’s default limit is actually 3000, but I’m very reticent to modify it instead of the code. Besides, changing the limit only postpones the problem to the next longer chain of comments :/
I suspect there's something else going on here, or your comments data is far different than I'm supposing.
Your flatten method works just fine for me with a tree of 600 comments. In fact, it works fine with 600,000 comments (recurs. lim. 1000)
Example test script:
https://pastebin.com/6Mdvu2WL
Without seeing your other code, I don't know what the issue you're running into. At a wild guess, is it possible your graph is not acyclic?
As far as I can tell, your data would have to be 3000 comments deep to trigger the recursion limit, and with 600 comments that just isn't possible without the graph pointing back into itself.
?
(6M comments worked as expected, though it took a while)
@cazabon I was also surprised by the trigger so early down the branch.
That _remove_recursion function is there to remove links to other elements in the tree (beside replies), to avoid loops and in fact there is no loop, I tested it.
I suspect it has to do with __iter__, which uses yield to return the elements. Each comment also has a field for the author, and dict is called on that as well. I think that somehow is adding to the stack even though it’s a different instruction.
@cazabon _remove_recursion simply goes down the branches from a given node to remove links to the parent object, the parent comment (if any), and is then called on each reply until it reaches the end of the branch.
@cazabon It’s 600 comments down the branch :)
So reply of a reply of a reply… etc. Not just 600 comments in a list.
Ah, not 600 comments, 600-deep comments. Gotcha.
@cazabon Yep, depth not width.
Still weird that it would fail so soon though, it threw the error 488 comments deep, just around 1/6 of the recursion limit of my machine.
I think dict and yield are doing something weird behind the scenes that keeps the stack from clearing, because I tested it and there is no looping: it fails at the first branch and does not deviate from the intended depth-first behaviour 