Add Cython acceleration for result set decode#197
Conversation
~9× speedup on large fetches by decoding wire-format batches in C instead of Python. Falls back to pure Python if the extension isn't built.
|
@TheDistributor please don't open your PR until it's done and tested and you're ready for it to be reviewed. Else we get spammed with email about every new commit :) |
|
This needs a larger discussion with @rshaull etc. There is the effort to use the C driver which seems to overlap with this. There are issues with supporting python 2, which I don't think this was tested with. There are tests that need to be made to ensure that we can still build and use the driver even if the user doesn't have all the cython infrastructure or a C compiler. There are discussions that need to happen about how we distribute the driver (via pypi or whatever) if it has a compiled component; do we have to build and upload wheels for various different platforms? Or...? |
2 participants
~9× speedup on large fetches by decoding wire-format batches in C instead of Python. Falls back to pure Python if the extension isn't built.