Output files detected with size 0 on planemo tests

[pcm32]

I have been running planemos tests this morning for some tools (latest planemo version), and for different output formats (png, H5, etc) outputs, when compared by size, I get that planemo detects 0 bytes sizes for output files. The command executes correctly and with a 0 exit code and takes some seconds to run (as expected). Manually executing the same command on the same conda environment with the same input files and parameters also runs correctly and the result files clearly have content (they are not empty).

"output_problems": [
                    "Output output_h5:  different than expected, difference (using sim_size):\n( /tmp/tmp8axe7ki2filter_genes.h5 v. /tmp/tmpo0xtgzmkfilter_genes.h5 )\nFiles /tmp/tmp8axe7ki2filter_genes.h5=3632967b but /tmp/tmpo0xtgzmkfilter_genes.h5=0b - compare by size (delta=10000) failed"
                ],

Galaxy logs show nothing out of the ordinary (I think):

galaxy.tools.execute DEBUG 2024-02-10 11:05:46,706 [pN:main.1,p:2084907,tN:WSGI_0] Created 1 job(s) for tool scanpy_filter_genes request (1515.525 ms)

galaxy.jobs.mapper DEBUG 2024-02-10 11:05:47,269 [pN:main.1,p:2084907,tN:JobHandlerQueue.monitor_thread] (2) Mapped job to destination id: planemo_des
t
galaxy.jobs.handler DEBUG 2024-02-10 11:05:47,336 [pN:main.1,p:2084907,tN:JobHandlerQueue.monitor_thread] (2) Dispatching to planemo_runner runner
galaxy.jobs DEBUG 2024-02-10 11:05:47,641 [pN:main.1,p:2084907,tN:JobHandlerQueue.monitor_thread] (2) Persisting job destination (destination id: plan
emo_dest)
galaxy.jobs DEBUG 2024-02-10 11:05:47,708 [pN:main.1,p:2084907,tN:JobHandlerQueue.monitor_thread] (2) Working directory for job is: /tmp/tmp0dnm7wzc/j
ob_working_directory/000/2

galaxy.jobs.runners DEBUG 2024-02-10 11:05:48,077 [pN:main.1,p:2084907,tN:JobHandlerQueue.monitor_thread] Job [2] queued (740.436 ms)
galaxy.jobs.handler INFO 2024-02-10 11:05:48,092 [pN:main.1,p:2084907,tN:JobHandlerQueue.monitor_thread] (2) Job dispatched
galaxy.security.object_wrapper WARNING 2024-02-10 11:05:48,237 [pN:main.1,p:2084907,tN:LocalRunner.work_thread-0] Unable to create dynamic subclass Sa
feStringWrapper__galaxy.model.none_like.None__NoneType__NotImplementedType__Number__SafeStringWrapper__ToolParameterValueWrapper__bool__bytearray__ell
ipsis for <class 'galaxy.model.none_like.NoneDataset'>, None: type() doesn't support MRO entry resolution; use types.new_class()
galaxy.jobs DEBUG 2024-02-10 11:05:48,346 [pN:main.1,p:2084907,tN:LocalRunner.work_thread-0] Job wrapper for Job [2] prepared (235.130 ms)
galaxy.tool_util.deps DEBUG 2024-02-10 11:05:48,347 [pN:main.1,p:2084907,tN:LocalRunner.work_thread-0] Using dependency scanpy-scripts version 1.9.0 o
f type conda
galaxy.tool_util.deps DEBUG 2024-02-10 11:05:48,348 [pN:main.1,p:2084907,tN:LocalRunner.work_thread-0] Using dependency scanpy-scripts version 1.9.0 o
f type conda
galaxy.jobs.command_factory INFO 2024-02-10 11:05:48,380 [pN:main.1,p:2084907,tN:LocalRunner.work_thread-0] Built script [/tmp/tmp0dnm7wzc/job_working
_directory/000/2/tool_script.sh] for tool command [[ "$(basename "$CONDA_DEFAULT_ENV")" = "$(basename '/home/ubuntu/miniconda3/envs/[email protected]')" ] || {
MAX_TRIES=3
COUNT=0
while [ $COUNT -lt $MAX_TRIES ]; do
    . '/home/ubuntu/miniconda3/bin/activate' '/home/ubuntu/miniconda3/envs/[email protected]' > conda_activate.log 2>&1
    if [ $? -eq 0 ];then
        break
    else
        let COUNT=COUNT+1
        if [ $COUNT -eq $MAX_TRIES ];then
            echo "Failed to activate conda environment! Error was:"
            cat conda_activate.log
            exit 1
        fi
        sleep 10s
    fi
done
} ; ln -s '/tmp/tmp0dnm7wzc/files/e/8/a/dataset_e8ad1ed7-c8d4-4b2d-bd0b-d1def84f03d5.dat' input.h5 && PYTHONIOENCODING=utf-8 scanpy-filter-genes --param 'g:n_cells' 20.0 1000000000.0 --param 'g:n_counts' 0.0 1000000000.0     --input-format 'anndata' input.h5   --show-obj stdout --output-format anndata output.h5]
galaxy.jobs.runners DEBUG 2024-02-10 11:05:48,493 [pN:main.1,p:2084907,tN:LocalRunner.work_thread-0] (2) command is: mkdir -p working outputs configs
if [ -d _working ]; then
    rm -rf working/ outputs/ configs/; cp -R _working working; cp -R _outputs outputs; cp -R _configs configs
else
    cp -R working _working; cp -R outputs _outputs; cp -R configs _configs
fi
cd working; /bin/bash /tmp/tmp0dnm7wzc/job_working_directory/000/2/tool_script.sh > '../outputs/tool_stdout' 2> '../outputs/tool_stderr'; return_code=$?; echo $return_code > /tmp/tmp0dnm7wzc/job_working_directory/000/2/galaxy_2.ec;
if [ -f "/tmp/tmp0dnm7wzc/job_working_directory/000/2/working/output.h5" ] ; then cp "/tmp/tmp0dnm7wzc/job_working_directory/000/2/working/output.h5" "/tmp/tmp0dnm7wzc/job_working_directory/000/2/outputs/dataset_497ce027-d2cc-4984-bebc-aa8e70d1654b.dat" ; fi; cd '/tmp/tmp0dnm7wzc/job_working_directory/000/2';
[ "$GALAXY_VIRTUAL_ENV" = "None" ] && GALAXY_VIRTUAL_ENV="$_GALAXY_VIRTUAL_ENV"; _galaxy_setup_environment True; python metadata/set.py; sh -c "exit $return_code"
galaxy.jobs.runners.local DEBUG 2024-02-10 11:05:48,516 [pN:main.1,p:2084907,tN:LocalRunner.work_thread-0] (2) executing job script: /tmp/tmp0dnm7wzc/job_working_directory/000/2/galaxy_2.sh

galaxy.jobs.runners.util.process_groups DEBUG 2024-02-10 11:05:54,924 [pN:main.1,p:2084907,tN:LocalRunner.work_thread-0] check_pg(): No process found in process group 2084973
galaxy.jobs.runners.local DEBUG 2024-02-10 11:05:54,925 [pN:main.1,p:2084907,tN:LocalRunner.work_thread-0] execution finished: /tmp/tmp0dnm7wzc/job_working_directory/000/2/galaxy_2.sh

galaxy.jobs DEBUG 2024-02-10 11:05:55,222 [pN:main.1,p:2084907,tN:LocalRunner.work_thread-0] finish(): Moved /tmp/tmp0dnm7wzc/job_working_directory/000/2/outputs/dataset_497ce027-d2cc-4984-bebc-aa8e70d1654b.dat to /tmp/tmp0dnm7wzc/files/4/9/7/dataset_497ce027-d2cc-4984-bebc-aa8e70d1654b.dat
galaxy.model.metadata DEBUG 2024-02-10 11:05:55,412 [pN:main.1,p:2084907,tN:LocalRunner.work_thread-0] loading metadata from file for: HistoryDatasetAssociation 2
galaxy.jobs DEBUG 2024-02-10 11:05:55,848 [pN:main.1,p:2084907,tN:LocalRunner.work_thread-0] job_wrapper.finish for job 2 executed (662.024 ms)

Shut down
supervisord has terminated
There were problems with 1 test(s) - out of 1 test(s) executed. See /home/ubuntu/container-galaxy-sc-tertiary/tools/tertiary-analysis/scanpy/scanpy-filter-genes_report.html for detailed breakdown.
scanpy_filter_genes (Test #1): failed

Galaxy version is end of Oct last year. This is on a linux machine (ubuntu). Thanks!

[mvdbeek]

If you want us to look into this can you please provide instructions for how to test this.

[pcm32]

Thanks. I'll try to reduce the particularities to a simpler example (this might uncover a clue as well) and post here.