search sys.path when checking if module is builtin / binary #283
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi
Not sure of the process to submit PR, so hope this is enough info etc. If not then please let me know!
I came across an issue when we make use of PYTHONPATH to pick up modules and these modules contain shared libraries
Example code
Output
/usr/local/python/3.6.2/bin/python3 d1.py
<class 'numpy.uint64'>
/u1/morrpat/workspace/libs/1.14.3-cp36/numpy/core/multiarray.cpython-36m-x86_64-linux-gnu.so
Traceback (most recent call last):
File "d1.py", line 16, in
f()
File "d1.py", line 14, in f
dill.dump(ui, f)
File "/u1/morrpat/workspace/libs/0.2.7.1/dill/dill.py", line 274, in dump
pik.dump(obj)
File "/usr/local/python/3.6.2/lib/python3.6/pickle.py", line 409, in dump
self.save(obj)
File "/usr/local/python/3.6.2/lib/python3.6/pickle.py", line 521, in save
self.save_reduce(obj=obj, *rv)
File "/usr/local/python/3.6.2/lib/python3.6/pickle.py", line 609, in save_reduce
save(func)
File "/usr/local/python/3.6.2/lib/python3.6/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
File "/u1/morrpat/workspace/libs/0.2.7.1/dill/dill.py", line 1033, in save_builtin_method
pickler.save_reduce(_get_attr, (module, obj.name), obj=obj)
File "/usr/local/python/3.6.2/lib/python3.6/pickle.py", line 610, in save_reduce
save(args)
File "/usr/local/python/3.6.2/lib/python3.6/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
File "/usr/local/python/3.6.2/lib/python3.6/pickle.py", line 736, in save_tuple
save(element)
File "/usr/local/python/3.6.2/lib/python3.6/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
File "/u1/morrpat/workspace/libs/0.2.7.1/dill/dill.py", line 1239, in save_module
state=_main_dict)
File "/usr/local/python/3.6.2/lib/python3.6/pickle.py", line 634, in save_reduce
save(state)
File "/usr/local/python/3.6.2/lib/python3.6/pickle.py", line 476, in save
f(self, obj) # Call unbound method with explicit self
File "/u1/morrpat/workspace/libs/0.2.7.1/dill/dill.py", line 871, in save_module_dict
StockPickler.save_dict(pickler, obj)
File "/usr/local/python/3.6.2/lib/python3.6/pickle.py", line 821, in save_dict
self._batch_setitems(obj.items())
File "/usr/local/python/3.6.2/lib/python3.6/pickle.py", line 847, in _batch_setitems
save(v)
File "/usr/local/python/3.6.2/lib/python3.6/pickle.py", line 496, in save
rv = reduce(self.proto)
TypeError: can't pickle PyCapsule objects
The fix has it check against all the paths in sys.path as well if it hasnt already determined it is a builtin module.