diff --git a/docs/documentation.md b/docs/documentation.md new file mode 100644 index 0000000000000000000000000000000000000000..6513a508996d96ef3d43bd7f471ac15c7795267d --- /dev/null +++ b/docs/documentation.md @@ -0,0 +1,1168 @@ +# F500 postprocessing tool +The F500 Data Analytics project provides a suite of tools and scripts for processing, analyzing, and visualizing point cloud data, particularly from Phenospex PLY PointCount files. + +#### Developed by Sven Warris + +- GitHub: [https://git.wur.nl/NPEC/analytics](https://git.wur.nl/NPEC/analytics) + +- Contact: [sven.warris@wur.nl](mailto:sven.warris@wur.nl) + +## Module rescaleWavelength.py +### rescale_wavelengths( ) + +Rescales the wavelengths of a point cloud data (PCD) object by a given scale factor. + +This function takes a PCD object with attributes for wavelengths, colors, and NIR values. +It rescales the wavelengths by dividing them by the provided scale factor and updates the +color and NIR attributes of the PCD accordingly. + +Parameters: +pcd : object + A point cloud data object that contains 'wavelengths', 'colors', and 'nir' attributes. + The 'wavelengths' attribute is expected to be a 2D array with columns representing + R, G, B, and NIR wavelengths. +scale : float + The scale factor by which to divide the wavelengths. + +Returns: +object + The modified PCD object with rescaled color and NIR attributes. + +Side Effects: +Modifies the 'colors' and 'nir' attributes of the input PCD object in place. + +Exceptions: +This function assumes that the input PCD object has the required attributes and that they +are in the expected format. If not, it may raise AttributeError or IndexError. + +Future Work: +Consider adding input validation to ensure the PCD object has the required attributes and +that they are in the expected format. Additionally, handle potential exceptions more gracefully. + + +--- + +## Module computePhenotypes.py +### get_ndvi_for_visualization( ) + +Calculate the NDVI for visualization purposes, scaling the result between 0 and 1. + +Parameters: +pcd (object): A point cloud data object containing 'wavelengths' and 'ndvi' attributes. + 'wavelengths' is expected to be a 2D array where the columns correspond to different spectral bands. + +Returns: +ndarray: A 1D array of NDVI values scaled between 0 and 1. + +Side Effects: +- Modifies the 'ndvi' attribute of the input 'pcd' object. + +Notes: +- This function ignores division and invalid operation warnings using numpy's seterr function. + + +### get_ndvi( ) + +Calculate the standard NDVI, with values ranging from -1 to 1. + +Parameters: +pcd (object): A point cloud data object containing 'wavelengths' and 'ndvi' attributes. + 'wavelengths' is expected to be a 2D array where the columns correspond to different spectral bands. + +Returns: +ndarray: A 1D array of NDVI values ranging from -1 to 1. + +Side Effects: +- Modifies the 'ndvi' attribute of the input 'pcd' object. + +Notes: +- This function ignores division and invalid operation warnings using numpy's seterr function. + + +### get_npci( ) + +Calculate the Normalized Pigment Chlorophyll Index (NPCI) using the red and blue channels. + +Parameters: +pcd (object): A point cloud data object containing 'wavelengths' attribute. + 'wavelengths' is expected to be a 2D array where the columns correspond to different spectral bands. + +Returns: +ndarray: A 1D array of NPCI values. + +Notes: +- This function ignores division and invalid operation warnings using numpy's seterr function. + + +### get_greenness( ) + +Calculate a greenness index using the red, green, and blue channels. + +Parameters: +pcd (object): A point cloud data object containing 'wavelengths' attribute. + 'wavelengths' is expected to be a 2D array where the columns correspond to different spectral bands. + +Returns: +ndarray: A 1D array of greenness index values. + +Notes: +- This function ignores division and invalid operation warnings using numpy's seterr function. + + +--- + +## Module histograms_ply.py +### createPNG( ) + +Creates a PNG image from a point cloud. + +Parameters: +pcd (open3d.geometry.PointCloud): The point cloud to visualize and capture. +filename (str): The filename where the image will be saved. + +Side Effects: +Opens a visualization window and saves a screenshot as a PNG file. + + +--- + +## Module animate_ply.py +### play_motion( ) + +Plays an animation of a list of point clouds using Open3D's visualization tools. + +This function initializes a visualizer and iterates through the provided list of point clouds, +updating the visualizer with each point cloud to create an animation effect. It captures each +frame as a PNG image in the /tmp directory. + +Args: +- list_of_pcds (list): A list of Open3D point cloud objects to be animated. + +Side Effects: + - Creates PNG images in the /tmp directory for each frame of the animation. + - Opens a visualization window to display the animation. + +Notes: + - The function uses a nested callback structure to handle forward and backward animation. + - Future work could include adding more control over the animation speed and direction. + + +### reset_motion( ) + +Resets the animation to the first frame. + +Args: +- vis: The Open3D visualizer instance. + +Returns: +- bool: Always returns False to indicate the animation should not stop. + + + +### backward( ) + +Moves the animation one frame backward. + +Args: +- vis: The Open3D visualizer instance. + + + +### forward( ) + +Moves the animation one frame forward and captures the frame as a PNG image. + +Args: +- vis: The Open3D visualizer instance. + +Returns: +- bool: Always returns False to indicate the animation should not stop. + + + +--- + +## Module visualization_ply.py +### main( ) + +Main function to execute the point cloud processing and visualization. + +This function reads a point cloud file specified by the command line argument, visualizes it, computes NDVI, +and visualizes the NDVI data. It also colorizes the NDVI data for enhanced visualization. + +Raises: +- IndexError: If no file path is provided as a command line argument. + + +- FileNotFoundError: If the specified file does not exist. + + +- AttributeError: If the point cloud object does not have the required methods. + + + +--- + +## Module clearWhites.py +### main( ) + +Main function to read a PLY file and load it into an Open3D PointCloud object. + +This function reads a PLY file specified as a command-line argument and loads it into a PointCloud object using the Open3D library. + +Raises: +- IndexError: If no command-line argument is provided for the PLY file path. + + +- FileNotFoundError: If the specified PLY file does not exist. + + +- Exception: If there is an error reading the PLY file. + +Side Effects: + Loads the point cloud data into memory. + +Future Work: + - Add error handling for unsupported file formats. + - Implement visualization of the point cloud data. + + +--- + +## Module PointCloud.py +### Class: PointCloud( ) + +A class to represent and manipulate a point cloud using Open3D. + +Attributes: +---------- +pcd : open3d.geometry.PointCloud + The point cloud data. +trimmed : bool + A flag indicating whether the point cloud has been trimmed. + + +### __init__( ) + +Initializes the PointCloud object by reading point cloud data from a file. + +Parameters: +---------- +filename : str + The path to the point cloud file in PLY format. + + +### writeHistogram( ) + +Writes a histogram of the given data to a file. + +Parameters: +---------- +data : numpy.ndarray + The data for which the histogram is to be calculated. +filename : str + The path to the file where the histogram will be written. +timepoint : str + The timepoint associated with the data. +sampleName : str + The name of the sample. +bins : int + The number of bins for the histogram. +dataRange : tuple, optional + The lower and upper range of the bins. If not provided, range is (data.min(), data.max()). + +Side Effects: +------------ +Writes the histogram data to the specified file. + + +### getWavelengths( ) + +Retrieves the wavelengths from the point cloud. + +Returns: +------- +numpy.ndarray + The wavelengths as a numpy array. If the point cloud is trimmed, returns a vertically stacked array. + + +### get_psri( ) + +Calculates the Plant Senescence Reflectance Index (PSRI). + +Returns: +------- +numpy.ndarray + The PSRI values calculated as (RED - GREEN) / NIR. + + +### get_hue( ) + +Calculates the hue from the RGB wavelengths. + +Returns: +------- +numpy.ndarray + The hue values calculated from the RGB wavelengths. + + +### get_greenness( ) + +Calculates the greenness index. + +Returns: +------- +numpy.ndarray + The greenness values calculated as (2*G - R - B) / (2*R + G + B). + + +### get_ndvi( ) + +Calculates the Normalized Difference Vegetation Index (NDVI). + +Returns: +------- +numpy.ndarray + The NDVI values calculated as (NIR - RED) / (NIR + RED). + + +### get_npci( ) + +Calculates the Normalized Pigment Chlorophyll Index (NPCI). + +Returns: +------- +numpy.ndarray + The NPCI values calculated as (RED - BLUE) / (RED + BLUE). + + +### setColors( ) + +Sets the colors of the point cloud. + +Parameters: +---------- +colors : numpy.ndarray + The colors to be set for the point cloud. + + +### render_image( ) + +Renders an image of the point cloud. + +Parameters: +---------- +filename : str + The path to the file where the image will be saved. +image_width : int + The width of the image. +image_height : int + The height of the image. +rescale : bool, optional + Whether to rescale the colors before rendering. Default is True. + + +### trim( ) + +Trims the point cloud based on the z-values. + +Parameters: +---------- +zIndex : float + The z-value threshold for trimming the point cloud. + +Side Effects: +------------ +Modifies the point cloud to only include points with z-values greater than or equal to zIndex. + + +### render_image_no_rescale( ) + +Renders an image of the point cloud without rescaling the colors. + +Parameters: +---------- +filename : str + The path to the file where the image will be saved. +image_width : int + The width of the image. +image_height : int + The height of the image. + +Side Effects: +------------ +Saves the rendered image to the specified file. + + +### render_image_rescale( ) + +Renders an image of the point cloud with rescaled colors. + +Parameters: +---------- +filename : str + The path to the file where the image will be saved. +image_width : int + The width of the image. +image_height : int + The height of the image. + +Side Effects: +------------ +Saves the rendered image to the specified file. + + +--- + +## Module deleteFAIRObject.py +### main( ) + +Main function to execute the deletion of resources. + +This function sets up the session with the necessary headers and iterates over predefined ranges +to delete resources from the server. It deletes data files, samples, assays, studies, and investigations. + +Raises: +- requests.exceptions.RequestException: If a network-related error occurs during the requests. + + + +--- + +## Module F500.py +### Class: F500( ) + +A class to handle the processing of F500 PlantEye data, including restructuring, point cloud processing, histogram combination, and data upload. + +Attributes: + description (defaultdict): A dictionary to store descriptions. + columnsToDrop (list): A list of columns to drop from the data. + ISA (dict): A dictionary to store ISA-related data. + datamatrix (list): A list to store data matrix information. + investigation (Investigation): An ISA investigation object. + checkAssayName (re.Pattern): A regex pattern to check assay names. + measurements (DataFrame): A DataFrame to store measurements. + currentFile (str): The current file being processed. + currentRoot (str): The current root directory being processed. + command (str): The command to execute. + assaysDone (set): A set to store completed assays. + samples (dict): A dictionary to store sample objects. + + +### __init__( ) + +Initializes the F500 object with default values and configurations. + + +### commandLineInterface( ) + +Sets up the command-line interface for the script, defining arguments and subcommands. + + +### setLogger( ) + +Configures the logging for the script based on command-line arguments. + + +### removeAfterSpaceFromDataMatrix( ) + +Cleans up the 'DataMatrix' column in a DataFrame row by removing text after a space. + +Args: +- row (Series): A row from a DataFrame. + +Returns: +- Series: The modified row with cleaned 'DataMatrix' column. + + + +### createISA( ) + +Initializes an ISA investigation object and sets up the study and metadata. + + +### writeISAJSON( ) + +Writes the ISA investigation object to a JSON file. + + +### copyPots( ) + +Copies pot information from a reference DataFrame to a row. + +Args: +- row (Series): A row from a DataFrame. + + +- pots (DataFrame): A DataFrame containing pot information. + + +- f500 (F500): An instance of the F500 class. + +Returns: +- Series: The modified row with pot information. + + + +### measurementsToFile( ) + +Writes the measurements DataFrame to a file. + + +### rawMeasurementsToFile( ) + +Writes raw measurements to a file. + +Args: +- path (str): The directory path to save the file. + + +- filename (str): The name of the file. + + +- measurements (list): A list of measurements to write. + + + +### addPointClouds( ) + +Adds point cloud file names to a row. + +Args: +- row (Series): A row from a DataFrame. + + +- title (str): The title to use in the file names. + +Returns: +- Series: The modified row with point cloud file names. + + + +### copyPointcloudFile( ) + +Copies point cloud files to a specified location. + +Args: +- row (Series): A row from a DataFrame. + + +- f500 (F500): An instance of the F500 class. + + +- fullPath (str): The destination path for the point cloud files. + + + +### copyPlotPointcloudFile( ) + +Copies plot point cloud files to a specified location. + +Args: +- row (Series): A row from a DataFrame. + + +- f500 (F500): An instance of the F500 class. + + +- fullPath (str): The destination path for the plot point cloud files. + + +- title (str): The title to use in the file names. + + + +### createSample( ) + +Creates a sample object if it doesn't already exist. + +Args: +- samples (dict): A dictionary to store sample objects. + + +- name (str): The name of the sample. + + +- source (Source): The source object for the sample. + + +- organism (str): The organism name. + + +- taxon (str): The taxon ID. + + +- term_source (OntologySourceReference): The ontology source reference. + +Returns: +- Sample: The created or existing sample object. + + + +### createAssay( ) + +Creates an assay object and adds it to the investigation. + +Args: +- row (Series): A row from a DataFrame. + + +- f500 (F500): An instance of the F500 class. + + +- path (str): The directory path for the assay. + + +- source (Source): The source object for the assay. + + + +### createAssayPlot( ) + +Creates an assay plot object and adds it to the investigation. + +Args: +- row (Series): A row from a DataFrame. + + +- f500 (F500): An instance of the F500 class. + + +- path (str): The directory path for the assay plot. + + +- source (Source): The source object for the assay plot. + + +- title (str): The title to use in the file names. + + + +### correctDataMatrix( ) + +Corrects the 'DataMatrix' column in a row based on a reference DataFrame. + +Args: +- row (Series): A row from a DataFrame. + + +- pots (DataFrame): A DataFrame containing pot information. + +Returns: +- Series: The modified row with corrected 'DataMatrix' column. + + + +### finalize( ) + +Finalizes the processing of measurements and creates assays. + +Args: +- title (str): The title to use in the file names. + + + +### getDirectoryListing( ) + +Returns a directory listing for a given root folder. + +Args: +- rootFolder (str): The root folder to list. + +Returns: +- generator: A generator yielding directory listings. + + + +### restructure( ) + +Restructures the raw data into an ISA-compliant format. + + +### processPointclouds( ) + +Processes point cloud files and generates derived data. + + +### combineHistograms( ) + +Combines histogram data from multiple assays into a single file. + + +### upload( ) + +Uploads the processed data to a specified platform. + + +--- + +## Module processPointClouds.py +### writeHistogram( ) + +Writes a histogram of the given data to a file. + +Parameters: + data (numpy.ndarray): The data for which the histogram is to be computed. + filename (str): The name of the file where the histogram will be written. + +Outputs: + A file containing the histogram data. The bin edges and histogram counts + are written in separate lines, separated by semicolons. + +Side Effects: + Creates or overwrites the specified file with histogram data. + + +### get_greenness( ) + +Calculates the greenness index for a point cloud. + +Parameters: + pcd (open3d.geometry.PointCloud): The point cloud object containing wavelength data. + +Returns: +- numpy.ndarray: An array of greenness values for each point in the point cloud. + +Exceptions: + May raise an exception if the point cloud does not contain wavelength data. + +Notes: + The greenness index is calculated using the formula: + (R - B + 2G) / (R + G + B), where R, G, and B are the red, green, and blue + wavelength values, respectively. + + +--- + +## Module Fairdom.py +### Class: Fairdom( ) + +A class to manage the creation and upload of investigations, studies, assays, samples, and data files to the FAIRDOM platform. + +Attributes: + investigation: An ISA-Tools investigation object containing the data to be uploaded. + args: Command-line arguments or configuration settings for the upload process. + logger: A logging object to record the process of uploading data. + session: A requests session object configured with headers for authentication with the FAIRDOM API. + + +### __init__( ) + +Initializes the Fairdom class with the given investigation, arguments, and logger. + +Args: +- investigation: An ISA-Tools investigation object. + + +- args: An object containing command-line arguments or configuration settings. + + +- logger: A logging object for recording the upload process. + +Side Effects: + Updates the session headers with authentication information. + + +### createInvestigationJSON( ) + +Creates a JSON structure for an investigation. + +Returns: +- A dictionary representing the JSON structure of the investigation. + + + +### createStudyJSON( ) + +Creates a JSON structure for a study. + +Args: +- study: An ISA-Tools study object. + + +- investigationID: The ID of the investigation to which the study belongs. + +Returns: +- A dictionary representing the JSON structure of the study. + + + +### createAssayJSON( ) + +Creates a JSON structure for an assay. + +Args: +- assay: An ISA-Tools assay object. + + +- studyID: The ID of the study to which the assay belongs. + +Returns: +- A dictionary representing the JSON structure of the assay. + + + +### createDataFileJSON( ) + +Creates a JSON structure for a data file. + +Args: +- data_file: An object representing a data file. + +Returns: +- A dictionary representing the JSON structure of the data file. + + + +### addSampleToAssayJSON( ) + +Adds a sample to an assay JSON structure. + +Args: +- sampleID: The ID of the sample to be added. + + +- assayJSON: The JSON structure of the assay to which the sample will be added. + + + +### addDataFileToAssayJSON( ) + +Adds a data file to an assay JSON structure. + +Args: +- data_fileID: The ID of the data file to be added. + + +- assayJSON: The JSON structure of the assay to which the data file will be added. + + + +### addDataFilesToSampleJSON( ) + +Adds data files from an assay to a sample JSON structure. + +Args: +- assayJSON: The JSON structure of the assay containing the data files. + + +- sampleJSON: The JSON structure of the sample to which the data files will be added. + + + +### createSampleJSON( ) + +Creates a JSON structure for a sample. + +Args: +- sample: An ISA-Tools sample object. + +Returns: +- A dictionary representing the JSON structure of the sample. + + + +### upload( ) + +Uploads the investigation, studies, assays, samples, and data files to the FAIRDOM platform. + +Side Effects: + Communicates with the FAIRDOM API to create and upload data structures. + Logs the process and any errors encountered. + +Raises: +- SystemExit: If an error occurs during the upload process that prevents continuation. + + + +--- + +## Module F500Azure.py +### Class: F500Azure( ) + +A class to manage Azure Blob Storage interactions for plant imaging experiments. + +This class extends the F500 class and provides additional methods to initialize +Azure connections, transfer data between source and target containers, and handle +experiment metadata. + + +### __init__( ) + +Initialize the F500Azure class with a specific experiment ID. + +Args: +- experimentID (str): The unique identifier for the experiment. + + + +### initAzure( ) + +Initialize Azure-related settings and metadata for the experiment. + +Args: +- environment (dict): A dictionary containing environment-specific settings. + + +- metadata (dict): Metadata related to the experiment. + + +- logger (Logger): Logger instance for logging information. + +Side Effects: + Sets various attributes related to the experiment and Azure configuration. + + +### connectToSource( ) + +Connect to the source Azure Blob Storage container. + +Args: +- sourceConnectionString (str): Connection string for the source Azure Blob Storage. + + +- sourceContainerName (str): Name of the source container. + + +- sourceBlobName (str): Name of the source blob. + +Side Effects: + Initializes the source blob service and container clients. + + +### connectToTarget( ) + +Connect to the target Azure Blob Storage container. + +Args: +- targetConnectionString (str): Connection string for the target Azure Blob Storage. + + +- targetContainerName (str): Name of the target container. + + +- targetBlobName (str): Name of the target blob. + +Side Effects: + Initializes the target blob service and container clients. + + +### writeISAJSON( ) + +Write the investigation data to a JSON file. + +Side Effects: + Creates a JSON file with the investigation data. + + +### measurementsToFile( ) + +Write the measurements data to a CSV file. + +Side Effects: + Creates directories and a CSV file with the measurements data. + + +### rawMeasurementsToFile( ) + +Write raw measurements data to a CSV file. + +Args: +- path (str): The directory path where the file will be saved. + + +- filename (str): The name of the file. + + +- measurements (dict): The measurements data to be written. + +Side Effects: + Creates directories and a CSV file with the raw measurements data. + + +### copyPointcloudFile( ) + +Copy pointcloud files to a specified directory. + +Args: +- row (dict): A dictionary containing pointcloud file names. + + +- f500 (F500): An instance of the F500 class. + + +- fullPath (str): The destination directory path. + +Side Effects: + Copies pointcloud files to the specified directory. + +Exceptions: + Raises an exception if file copying fails. + + +### getDirectoryListing( ) + +Get a directory listing for the specified root folder. + +Args: +- rootFolder (str): The root folder path. + +Returns: +- generator: A generator yielding directory paths, directory names, and file names. + + + +--- + +## Module fairdom.py +### removeAfterSpaceFromDataMatrix( ) + +Removes any text after a space in the 'DataMatrix' column of a row. + +Args: +- row (pandas.Series): A row from a DataFrame. + +Returns: +- pandas.Series: The modified row with updated 'DataMatrix' value. + + + +### copyPots( ) + +Copies pot information from a pots DataFrame to a row based on matching coordinates. + +Args: +- row (pandas.Series): A row from a DataFrame. + + +- pots (pandas.DataFrame): A DataFrame containing pot information. + +Returns: +- pandas.Series: The modified row with updated pot information. + + + +### measurementsToFile( ) + +Saves the measurements DataFrame to a CSV file. + +Args: +- investigation (dict): The investigation dictionary. + + +- path (str): The directory path where the file will be saved. + + +- filename (str): The name of the file. + + +- measurements (pandas.DataFrame): The DataFrame containing measurements. + +Side Effects: + Creates directories and writes a CSV file to the specified path. + + +### rawMeasurementsToFile( ) + +Saves the raw measurements to a CSV file. + +Args: +- investigation (dict): The investigation dictionary. + + +- path (str): The directory path where the file will be saved. + + +- filename (str): The name of the file. + + +- measurements (pandas.DataFrame): The DataFrame containing raw measurements. + +Returns: +- str: The full path to the saved file. + +Side Effects: + Creates directories and writes a CSV file to the specified path. + + +### addPointClouds( ) + +Adds a point cloud filename to a row based on its coordinates and timestamp. + +Args: +- row (pandas.Series): A row from a DataFrame. + + +- title (str): The title used in the filename. + +Returns: +- pandas.Series: The modified row with the point cloud filename added. + + + +### createAssay( ) + +Creates an assay and uploads the associated data file to the SEEK platform. + +Args: +- row (pandas.Series): A row from a DataFrame containing assay data. + + +- investigation (dict): The investigation dictionary. + + +- path (str): The directory path for saving files. + + +- study_id (str): The ID of the study to which the assay belongs. + +Side Effects: + Creates directories, writes files, and uploads data to the SEEK platform. + + +### finalize( ) + +Finalizes the processing of measurements by creating assays and saving data files. + +Args: +- investigation (dict): The investigation dictionary. + + +- measurements (pandas.DataFrame): The DataFrame containing measurements. + + +- investigationPath (str): The directory path for saving files. + + +- title (str): The title used in filenames. + + +- metadata (pandas.DataFrame): The DataFrame containing metadata. + + +- study_id (str): The ID of the study to which the assays belong. + +Side Effects: + Creates directories, writes files, and uploads data to the SEEK platform. + + +--- + +## Module toolkit.py +### main( ) + +Main function to execute the command-line interface for the F500 class. + +This function initializes an instance of the F500 class and executes a method +based on the command-line arguments provided by the user. + +Side Effects: + Executes a method of the F500 class based on user input. + +Raises: +- Exception: If an invalid command is provided. + + + +--- + diff --git a/docs/documentation_onboarding_ai.md b/docs/documentation_onboarding_ai.md new file mode 100644 index 0000000000000000000000000000000000000000..5610f8fe3458487ac2837faf3359ba356b2a2582 --- /dev/null +++ b/docs/documentation_onboarding_ai.md @@ -0,0 +1,51 @@ +# Onboarding Guide for New Developers + +Welcome to the F500 Data Analytics project! This guide will help you get started with understanding the codebase, focusing on key modules, and following the typical development workflow. The F500 project is designed to process, analyze, and visualize point cloud data, particularly from Phenospex PLY PointCount files. + +## Codebase Structure + +The codebase is organized into several modules, each serving a specific purpose in the data processing pipeline. Below is an overview of the key modules: + +1. **rescaleWavelength.py**: Contains functions to rescale wavelengths in point cloud data. +2. **computePhenotypes.py**: Provides functions to compute various indices like NDVI, NPCI, and greenness for phenotypic analysis. +3. **histograms_ply.py**: Handles the creation of PNG images from point clouds. +4. **animate_ply.py**: Manages animations of point clouds using Open3D visualization tools. +5. **visualization_ply.py**: Main function for executing point cloud processing and visualization. +6. **clearWhites.py**: Reads PLY files and loads them into Open3D PointCloud objects. +7. **PointCloud.py**: Defines a class for representing and manipulating point clouds. +8. **deleteFAIRObject.py**: Manages the deletion of resources from a server. +9. **F500.py**: Core class for processing F500 PlantEye data, including restructuring, point cloud processing, and data upload. +10. **processPointClouds.py**: Contains functions for writing histograms and calculating greenness indices. +11. **Fairdom.py**: Manages the creation and upload of data to the FAIRDOM platform. +12. **F500Azure.py**: Extends the F500 class for Azure Blob Storage interactions. +13. **fairdom.py**: Provides utility functions for handling data matrices and measurements. +14. **toolkit.py**: Main function for executing the command-line interface for the F500 class. + +## Key Modules to Focus On + +As a new developer, you should focus on understanding the following key modules: + +- **F500.py**: This is the core module for processing PlantEye data. It includes methods for restructuring data, processing point clouds, and uploading data. +- **PointCloud.py**: This module provides a class for handling point cloud data, including methods for calculating various indices and rendering images. +- **computePhenotypes.py**: Understanding this module will help you grasp how different phenotypic indices are calculated from point cloud data. +- **Fairdom.py**: This module is crucial if you will be working on data uploads to the FAIRDOM platform. + +## Typical Development Workflow + +1. **Setup**: Clone the repository from [GitHub](https://git.wur.nl/NPEC/analytics) and set up your development environment. Ensure you have all necessary dependencies installed. + +2. **Understanding the Data**: Familiarize yourself with the point cloud data format and the specific attributes used in the project (e.g., wavelengths, colors, NIR values). + +3. **Feature Development**: When adding new features or modifying existing ones, focus on the relevant module. For example, if you're working on data visualization, you might focus on `visualization_ply.py` or `animate_ply.py`. + +4. **Testing**: Write unit tests for your code to ensure it works as expected. Pay attention to edge cases and potential exceptions, as noted in the documentation. + +5. **Documentation**: Update the documentation to reflect any changes you make. This includes updating docstrings and any relevant markdown files. + +6. **Code Review**: Submit your changes for code review. Engage with feedback to improve the quality of your code. + +7. **Deployment**: Once your changes are approved, follow the deployment process to integrate your changes into the main codebase. + +8. **Communication**: If you have questions or need assistance, reach out to Sven Warris at [sven.warris@wur.nl](mailto:sven.warris@wur.nl). + +By following this guide, you'll be well-equipped to contribute effectively to the F500 Data Analytics project. Welcome aboard! \ No newline at end of file diff --git a/docs/documentation_summary_ai.md b/docs/documentation_summary_ai.md new file mode 100644 index 0000000000000000000000000000000000000000..8472105ef721a71eb8e7db26070b8d6e186ee3ae --- /dev/null +++ b/docs/documentation_summary_ai.md @@ -0,0 +1,65 @@ +The F500 postprocessing tool is a comprehensive suite designed for processing, analyzing, and visualizing point cloud data, particularly from Phenospex PLY PointCount files. Developed by Sven Warris, this toolset is structured into several modules, each with distinct responsibilities and functionalities. Here's a summary of the key functionalities, workflows, and interactions among the modules: + +### Key Modules and Their Responsibilities: + +1. **rescaleWavelength.py**: + - **Functionality**: Rescales the wavelengths of a point cloud data (PCD) object by a given scale factor, updating the color and NIR attributes accordingly. + - **Interaction**: Directly modifies the PCD object, assuming it has specific attributes. + +2. **computePhenotypes.py**: + - **Functionality**: Computes various indices like NDVI, NPCI, and greenness for visualization and analysis. + - **Interaction**: Modifies the PCD object to include computed indices, using numpy for error handling. + +3. **histograms_ply.py**: + - **Functionality**: Creates PNG images from point clouds for visualization. + - **Interaction**: Utilizes Open3D for rendering and saving images. + +4. **animate_ply.py**: + - **Functionality**: Animates a list of point clouds, capturing each frame as a PNG. + - **Interaction**: Uses Open3D's visualization tools, with potential for future enhancements in animation control. + +5. **visualization_ply.py**: + - **Functionality**: Main function for executing point cloud processing and visualization, including NDVI computation. + - **Interaction**: Handles file input/output and visualization tasks. + +6. **clearWhites.py**: + - **Functionality**: Loads PLY files into Open3D PointCloud objects. + - **Interaction**: Focuses on file reading and error handling. + +7. **PointCloud.py**: + - **Functionality**: Represents and manipulates point cloud data, offering methods for index calculations and rendering. + - **Interaction**: Provides foundational operations for other modules to build upon. + +8. **deleteFAIRObject.py**: + - **Functionality**: Deletes resources from a server, iterating over predefined ranges. + - **Interaction**: Manages network requests and error handling. + +9. **F500.py**: + - **Functionality**: Central class for processing F500 PlantEye data, including restructuring, point cloud processing, and data upload. + - **Interaction**: Integrates various functionalities, managing data flow and processing logic. + +10. **processPointClouds.py**: + - **Functionality**: Writes histograms and calculates greenness indices for point clouds. + - **Interaction**: Focuses on data analysis and file output. + +11. **Fairdom.py**: + - **Functionality**: Manages the creation and upload of investigations, studies, assays, samples, and data files to the FAIRDOM platform. + - **Interaction**: Handles data structure creation and API communication. + +12. **F500Azure.py**: + - **Functionality**: Manages Azure Blob Storage interactions for plant imaging experiments, extending the F500 class. + - **Interaction**: Facilitates data transfer and storage in Azure environments. + +13. **toolkit.py**: + - **Functionality**: Provides a command-line interface for executing F500 class methods. + - **Interaction**: Acts as an entry point for user interaction with the toolset. + +### Unique Features and Design Patterns: + +- **Modular Design**: The toolset is divided into distinct modules, each handling specific tasks, promoting separation of concerns and ease of maintenance. +- **Error Handling**: Several modules employ numpy's error handling features to manage division and invalid operation warnings gracefully. +- **Open3D Integration**: Utilizes Open3D for point cloud visualization and manipulation, providing robust 3D data handling capabilities. +- **ISA-Compliant Data Structuring**: The F500 module restructures data into an ISA-compliant format, facilitating standardized data management and sharing. +- **FAIRDOM and Azure Integration**: Modules like Fairdom.py and F500Azure.py extend the tool's capabilities to interact with external platforms for data management and storage. + +Overall, the F500 postprocessing tool offers a comprehensive and flexible framework for handling point cloud data, with robust visualization, analysis, and data management features. \ No newline at end of file