Open the sorting reasult by phy

[BEGINRX]

When I finish sorting, try to fix manually. But there are some problem.
phy template-gui params.py --debug

14:07:20.546 [D] __init__:68          Start capturing exceptions.
14:07:20.553 [D] model:619            Loading spike clusters.
14:07:20.556 [D] model:569            No channel shank file found.
14:07:20.556 [D] model:692            Loading templates.
14:07:20.559 [D] model:720            Templates are sparse.
14:07:20.559 [W] model:667            Skipping spike waveforms that do not exist, they will be extracted on the fly from the raw data as needed.
14:07:20.560 [D] model:730            Loading the whitening matrix.
14:07:20.560 [D] model:434            Whitening matrix file not found.
14:07:20.560 [D] model:737            Loading the inverse of the whitening matrix.
14:07:20.562 [D] model:766            Loading features.
14:07:20.569 [E] __init__:62          An error has occurred (RuntimeError): ndarray subclass __array_wrap__ method returned an object which was not an instance of an ndarray subclass
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\admin\miniconda3\envs\phy2\Scripts\phy.exe\__main__.py", line 7, in <module>
    sys.exit(phycli())
             ^^^^^^^^
  File "C:\Users\admin\miniconda3\envs\phy2\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\admin\miniconda3\envs\phy2\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\Users\admin\miniconda3\envs\phy2\Lib\site-packages\click\core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\admin\miniconda3\envs\phy2\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\admin\miniconda3\envs\phy2\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\admin\miniconda3\envs\phy2\Lib\site-packages\click\decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\admin\miniconda3\envs\phy2\Lib\site-packages\phy\apps\__init__.py", line 159, in cli_template_gui
    template_gui(params_path, **kwargs)
  File "C:\Users\admin\miniconda3\envs\phy2\Lib\site-packages\phy\apps\template\gui.py", line 209, in template_gui
    model = load_model(params_path)
            ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\admin\miniconda3\envs\phy2\Lib\site-packages\phylib\io\model.py", line 1433, in load_model
    return TemplateModel(**get_template_params(params_path))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\admin\miniconda3\envs\phy2\Lib\site-packages\phylib\io\model.py", line 339, in __init__
    self._load_data()
  File "C:\Users\admin\miniconda3\envs\phy2\Lib\site-packages\phylib\io\model.py", line 460, in _load_data
    self.sparse_features = self._load_features()
                           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\admin\miniconda3\envs\phy2\Lib\site-packages\phylib\io\model.py", line 780, in _load_features
    cols = self._read_array(self._find_path('pc_feature_ind.npy'), mmap_mode='r')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\admin\miniconda3\envs\phy2\Lib\site-packages\phylib\io\model.py", line 488, in _read_array
    return read_array(path, mmap_mode=mmap_mode).squeeze()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: ndarray subclass __array_wrap__ method returned an object which was not an instance of an ndarray subclass

QWidget: Must construct a QApplication before a QWidget

But other results can be imported into phy successfully. I find the number of cluster is one. Does the single cluster affect the process?

[JoeZiminski]

Hey @BEGINRX, seems like the problem is in the read_array() function of phylib. As mmap_mode is set to "r", this should just be loading the file out = np.load(path, mmap_mode=mmap_mode) where the path is the path to 'pc_feature_ind.npy', and returning it directly. I am not certain from the final error message, but this loaded object may not be for some reason an ndarray, then applying squeeze() on it is erroring out.

Can you try in in python my_file = np.load(...path to 'pc_feature_ind.npy') and see what the my_file object is?

What sorter are you using? It is suspicious that there is only 1 unit, in this case maybe something has gone wrong in the 1-unit case with the saving of the data to an unexpected format.

[BEGINRX]

Thank you for your reply! I have checked the array in pc_feature_ind.npy. The data is array([[0]], dtype=int64). I used the mountainsorter5 sorter. I don't know why there is only one unit. Because in the similar condition in the next recording, there are many units. I will try to sort using other methods. Thank you again!

[JoeZiminski]

Weird, I don't see why that would error! Sorry I made a mistake in my last post, could you try:

out = np.load(...path to 'pc_feature_ind.npy', mmap_mode="r")

I think the next step if you are interested is to break into the phylib read_array function (you can find where it is installed with pip show phylib) and run it line-by-line to see exactly why the error is occuring. If you are interested but are not sure how I can tell you how, but it is a bit more hands on, so feel free to trying other methods to sort!

[BEGINRX]

I used mmap_mode. The result is following.

>>> data = np.load('pc_feature_ind.npy', mmap_mode='r')
>>> data
memmap([[0]], dtype=int64)

It's similar to the previous result. Does it mean there is no feature extracted? I think I will run it line-by-line when I have free time.
Thank you!

[JoeZiminski]

cool thanks, this seems possibly like a numpy bug! or something I am misunderstanding, but this should really work, and I think is due to there only being one unit.

import numpy as np

bad_array = np.array([0])
np.save("bad_array.npy", bad_array)
load_bad_array = np.load("bad_array.npy", mmap_mode="r")
load_bad_array.squeeze()  # get error

good_array = np.array([0, 0])
np.save("good_array.npy", good_array )
load_good_array  = np.load("good_array.npy", mmap_mode="r")
load_good_array .squeeze()  # no error

99% sure it is a numpy bug because you dont get the same behaviour with a normal array. Feel free to open an issue at numpy github.

[JoeZiminski]

If you want to load the data still there are workarounds by editing the phy code on your machine to handle the error, let me know if you have any questions about this.

[BEGINRX]

It's funny. Thanks for your help. I will try it.