-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pair developing HDF5 #181 #250
Conversation
1864c00
to
113f9ca
Compare
.github/workflows/ci.yml
Outdated
if: ${{ contains(matrix.os, 'ubuntu') }} | ||
run: | | ||
sudo apt-get update | ||
sudo apt-get install libhdf5-dev |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems to work, but we may want to statically link when compiling. (Actually, I thought we were doing this...?)
tdms/include/hdf5_io.h
Outdated
// get the dataset and dataspace | ||
H5::DataSet dataset = file_->openDataSet(dataset_name); | ||
H5::DataSpace dataspace = dataset.getSpace(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Need to handle the case where this is a H5::Group
, I think.
Co-authored-by: Mosè Giordano <[email protected]> Co-authored-by: Will Graham <[email protected]>
Basic class structure and some failing tests for TDD.
…se of stupid, not TDD.
With unfortunate duplication of memory into buffers. To be fixed in the future with flattened classes.
* Trying to decipher structs * Further towards reading a struct * Bypass the stack-smash bug. MATLAB saves character arrays as uint16s, and HDF5 is unable to interpret them as chars. As such, we need to do manual conversion in the test. * Rename binary and update contents
…ility (#280) * Reorganise hdf5 unit tests - Move uint16 to char/str functions to the unit test utils file - Create hdf5_and_tdms_objects subdirectory of unit/ to hold unit tests on interaction between hdf5 and tdms classes - Move the Matrix<double> test from test_hdf5_io into the new subdirectory - Data files needed for unit tests are defined in a unit_test_utils namespace to avoid redefinition across multiple files * Create file to test interface and hdf5 interactions - Add docstrings to interface.h since these are missing and I've just had to work out what they do - Create a matlab script that can reproduce the class_data.mat file which the hdf5 unit tests will try to create tdms objects from - Create the barebones test_hdf5_interface file * HDF5Reader can read from .mat file and produce an InterfaceComponent * File restructure: accounting for how many tests we are going to have with HDF5 * Prune includes * Add .mat file for HDF5-TDMS-object unit tests to run. - Adds scripts to reproduce this data, so in theory a new user can run a short MATLAB script to reproduce this - Had a play with trying to get setup-matlab to run these scripts before the unit tests, but alas, no.
* Skeleton for new method * FrequencyVectors use std::vectors instead of our custom object as members. [DOESN'T COMPILE YET] - FrequencyVectors is now just a struct that uses std::vector datatypes - SimualationManager is broken, along with most parts of the codebase that use FrequencyVectors - Pending update to allow code to COMPILE * Allow TDMS to actually be compiled again. [TESTS STILL FAIL, READER METHOD NOT READY YET] - Move .mat data generation scripts for unit tests into benchmarking folder - Adjust paths to unit test data accordingly - InputMatrices stores the filename so that we can minimise disruption as we transition away from MATLAB * IT WORKS * Create the abstract H5Dimensions class to ease reading objects. - Update HDF5Base::shape_of to return instances of this class - Add unit tests for class - Update HDF5Reader method to avoid recycled code * Remove MATLAB pointers to InterfaceComponents in initialisation. - simulation_manager can now instantate these members using HDF5Reader rather than still relying on the old InputMatrices object * Add docstrings to files touched * Produce unit test binaries on CI rather than track in repo * - Detaches MATLAB from Cuboid class. - Turns Cuboid class into a struct because MATLAB is now gone. - Adds failure-case checks to HDF5Reader::read() functions. Squashed commit of the following: commit ce8d15e311e9dbb93bd7d7fbcd6c9cffe69657e3 Author: willGraham01 <[email protected]> Date: Fri May 5 15:19:51 2023 +0100 Rename shapes.h to cuboid.h because that's all it contains commit fe83f568c3da7491d450693574975df90a3f15e1 Author: willGraham01 <[email protected]> Date: Fri May 5 15:13:48 2023 +0100 Fix faulty logic thrown up by test commit b37a89f031519c6a356b50f91a12ad8867b78333 Author: willGraham01 <[email protected]> Date: Fri May 5 15:01:58 2023 +0100 Allow tdms to actually compile commit d7c982b9165e3ae4ccf3f42e0fb8c5cdde55b644 Author: willGraham01 <[email protected]> Date: Fri May 5 14:31:43 2023 +0100 Write unit test and read method for Cuboid. Add failure test for FrequencyVectors - Add new .m script to produce an input file with bad object data - Update unit_test_utils with this new filepath - Write HDF5Reader(Cuboid *cube) method and unit test - Add failure-test check for FrequencyVectors using bad data file commit cf61970d2052ab02ae23822f0edd28e89044734f Author: willGraham01 <[email protected]> Date: Fri May 5 14:12:11 2023 +0100 Change Cuboid to be a struct because a class is overkill * Use one master script for setting up .mat files for unit tests. - setup_unit_tests.m now calls all the other data-generating scripts in the folder to save updating the ci.yaml each time.
* Rework DispersiveMultiLayer class into a struct. TDMS builds. - xyz_vector struct of 3 vector<double>s introduced - DispersiveMultiLayer is now a struct with xyz_vector members - Preserve method for checking if the medium is dispersive - Temporarily disable unit tests for DispersiveMultiLayer class - HDF5Reader::read() can now assemble a DispersiveMultiLayer - HDF5Reader has a method for reading directly into a vector<T> - H5Dimension can be passed a H5::DataSet as well as a H5::DataSpace * Update paths now we need HDF5 data as well as matlab data * TDMS builds and tests HDF5Reader::read_data_from_group (passes, no) * Fix seg-fault in read_dataset_from_group * Add DispersiveMultiLayer test * Update DispersiveMultiLayer tests to not be entangled with the MATLAB jargon * Update the CI so things actually work * Move content of benchmark_scripts readme into doc/developers * Update CI to produce unit test `.mat` and `.hdf5` data (#289) * Move content of benchmark_scripts readme into doc/developers * Update CI attempt 1 * Actually use the right syntax * Force python version to prevent MacOS using python2.7 * Rename files to better match the data they produce * Aplease windows syntax * Remove pip cache due to known windows bug: actions/setup-python#436 * Apply suggestions from code review
05ef9fc
to
30899d5
Compare
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## main #250 +/- ##
=====================================
+ Coverage 25% 48% +24%
=====================================
Files 65 72 +7
Lines 3627 7931 +4304
=====================================
+ Hits 893 3817 +2924
- Misses 2734 4114 +1380
☔ View full report in Codecov by Sentry. |
Cherry pick and put this into
Made it ready for review, then decided to not do that since the merge queue will eat it up. |
Resolves #181. Relates to #70.