a g:Y@sdZddlZddlZddlZddlmZddlmZ ddl m Z ddl Z ddlmZddlZddlmZmZmZmZmZddlZddZGdd d eZd!ddZd"ddZd#ddZddZd$ddZ d%ddZ!e"dkreZ#ej$%dee&d ee#e 'dS)&z[ Created on Mon Feb 13 16:05:41 2023 @author: Albert Test function for the multiset CCA N) Rectangle)eigh) OUTPUT_PATHTRACKED_FIBERS_PATH SESSION_NAMES SESSION_TIMESSESSION_REPEATScCsdSNr r r FD:\MPI_Brain\Codes\aversive_scripts_2025\aversive_scripts_2025\mCCA.pymainsr c@s6eZdZddZd ddZddZdd Zd d ZdS) MultisetCCAcCsdSr r selfr r r __init__#szMultisetCCA.__init__Nc st_fddtjD}tt|dkrBt|tdfddtjD}tt|dkrrtddjd_|dkrj}|_t jjfd }t jjfd }tjD]}tjD]}t ||j } | ||j|dj|j|djf<||kr| ||j|dj|j|djf<qqdjd||} |} t | | d \} t | } t jdd | dj_fd dtjD_fd dtjD_jjfS)z dataset_list: list of datasets, each assumed to be of size (N features) X (N samples) feature number must be the same across datasets, but samples can be different If cca_dim is None, use all. Otherwise must be int. csg|]}|jdqS)rshape.0m dataset_listr r -z#MultisetCCA.fit..z1Not all datasets have the same number of featurescsg|]}|jdqS)rrrrr r r2rz0Not all datasets have the same number of samplesrN)baxiscs2g|]*}|j|djdjfqS)rN)featuresr)eigvecsrr r rRrcsg|]}tjj|qSr )nplinalginvinv_transf_listrrr r rSr)lenMrangesetprint ValueErrorrrcca_dimr!zerosmatmulTrflipeigvalsr$ transf_list) rrr+ features_list samples_listRRDDm1m2RjkABr0r )rr rr fit&s: 48 zMultisetCCA.fitcCs|j||S)zl Transform from the specified dataset's space to canonical. Data must be (N features) X (N samples) )r1)rdatarr r r to_canonicalWszMultisetCCA.to_canonicalcCs|j||S)z Transform from canonical space to the specified dataset's space Canonical data must be of size (cca_dim) X (N samples))r$)rcanonical_datarr r r from_canonical\szMultisetCCA.from_canonicalcCs|||||S)zr Transform data from dataset m1's space to dataset m2's space data must be (N_features) X (N_samples) )r?r=)rr<r6r7r r r alignaszMultisetCCA.align)N)__name__ __module__ __qualname__rr;r=r?r@r r r r r !s  1r positionTFc st||rtD]}||} t|} tdtj} tdtj} tjj| | dtdd} t t| j d}|| d|d| }| |||<qg}gtD]L}||} t|} t j | | ||d|dd \}}}| | |qt||d \}dj dt\}}|dkr|d krfd d tD}fddtD|d<|}n|dkr|dkr҇fdd tD}fddtD|d<|}n|d kr&|d kr&fdd tD}fddtD|d<|}nd|d kr|dkrt||d \}fdd tD}fddtD|d<|}||fS)a Perform multiset CCA on "raw" M datasets. They are allowed to be of different sizes, so trimming, warping, etc. must be performed. Because of this, data warping (making each trial the same size, with each trial defined as passing the max_pos mark) is performed as well INPUT: pos_list: list of M position arrays, of size samples1, samples2, etc. data_list: list of M data arrays, each of size (i features) X (j samples) Note that neither features nor samples are assumed to be equal across datasets. They will be trimmed to account for that max_pos : maximum value that can be taken by position, we assume periodicity warping_bins : each trial will be split into this amount of bins warp_based_on : 'position' or 'time', type of warping (see warping() for details) return_warped_data : if True, returns the data warped to have the same number of bins per trial. return_trimmed_data : if True, returns the data trimmed so each session has the same number of trials. shuffle: randomly shifts each session independently OUTPUT: pos_list_aligned : the position values of the aligned data data_dict_aligned : {dataset index : [list of datasets aligned to it]} mCCA: multiset CCa object (includes transformation matrices as attributes) g?g?r)sizedtyperN averagingT)max_poswarp_sampling_type warp_based_onreturn_flattenedrJFcs*i|]"fddtDqS)cs&g|]}|d|qSr r@rr7)r+data_list_warpedr6mCCAr r rr2perform_warped_mCCA...r'r)r&r+rQrRr6r rz'perform_warped_mCCA..cs$g|]}|d|qSr r=r)r+rQrRr r rrz'perform_warped_mCCA.. canonicalcs(i|] fddtDqS)csg|]}||qSr rOrP)data_list_trimmedr6rRr r rrrSrTrU)r&rZrRrVr rWrcsg|]}||qSr rXr)rZrRr r rrcs*i|]"fddtDqS)cs&g|]}|d|qSr rOrP)r+ data_listr6rRr r rrrSrTrU)r&r+r[rRrVr rWrcs$g|]}|d|qSr rXr)r+r[rRr r rrcs*i|]"fddtDqS)cs&g|]}|d|qSr rOrP)r+rZr6rRr r rrrSrTrU)r&r+rZrRrVr rWrcs$g|]}|d|qSr rXr)r+rZrRr r rr)r%r'r!copyintpparamMAX_POSrandomrandintlistrpfwarpingappendtrim_dataset_listr r;)pos_listr[rJ warping_binsrLreturn_warped_datareturn_trimmed_datashufflerposr< min_shift max_shift random_shiftidxspos_list_warped pos_warped data_warped_pos_list_trimmedr1r$data_dict_alignedpos_list_alignedr )r&r+r[rZrQrRr perform_warped_mCCAgsP    rxc Cstj}t|D]\}}|jd}|dur.|}nt|tttjfvrLt|}njt|tfvr|durjt d||}d|vrt|d| d} t || d}nt|} t || }t |||g}qt|}g} t|D]\}}| |d|q| S)a Given a list of PCA arrays, reduce all to the SAME specified pca_dim. e.g. if specified dimension if 10, and one pca has only 8 components, then ALL will be reduced to 8 pca_list: list of PCA arrays, each of dimension "num features" X "num timepoints" pca_dim: specifies dimensions to keep -if None, take all -if int, take that many -if 'X%', where "X" is a scalar, take dimensions that explain at least X% of the variance for each PCA (needs variance explained list) -if 'X', where "X" is a float between 0 and 1, interpret it as taking X*100% of the variance (needs variance explained list) variance_explained_list: list of variance explained (output from scikit learn's PCA) rNzEPlease specify variance explained if "pca_dim" is set to a proportion%d)r!inf enumeratertyper]floatint32str TypeErrorfindrcdimensions_to_explain_varianceminre) pca_listpca_dimvariance_explained_listnum_components_to_setpca_idxpcacurrent_num_componentsnum_componentsvariance_explainedvariance_to_explainpca_list_reducedr r r set_dimension_of_pca_lists,    rcst|}fdd}||tfddt|D}g}g}t|D]d}|} | dd} | |dd} ||| | } || ||dd| | f} || qH||fS)z< Trim datasets to have same number of rounds and dimensions csfdd|DS)Ncsg|]}tj|dddqS)T)rJfull_rounds_only pos_thresh)rcget_round_endtimes_pairs)rrlrNr r rrz7trim_dataset_list....r )rgrNr r rz#trim_dataset_list..csg|]}t|qSr )r%)ri)round_endtimes_listr r rrz%trim_dataset_list..rrN)r%r!rr're)rgr[rJr&get_endtimes_from_list min_roundsrurZrstartend_tuplesstart_first_round end_min_round pos_trimmed data_trimmedr )rJrr rfs     rfcCsdd|D}|S)zs Used on the output of perform_warped_mCCA. Adds the canonical space. Normalizes all PCAs individually cSs i|]\}}|dd|DqS)cSsg|]}tj|dddqS)rrr)rcnormalize_data)rrr r r rrz9normalize_pca_dict_aligned...r )rkrr r r rWrz.normalize_pca_dict_aligned..)items)pca_dict_alignedrRpca_dict_aligned_normalizedr r r normalize_pca_dict_alignedsrsseWienerc Cs~t|}g} g} t|D]^} || || } } tj| | |||d\}}}| |tj| | d||d\}}}| |qt||f}t|D]b}| |}t|D]L}||kr| |}n*||||} } tj| | |d|d\}}||||f<qqt||f}t|D]l}| |}t|D]T}||kr4| |}n.|||||} } tj| | |d|d\}}||||f<qq||fS)a Given M sets of data, return the prediction errors when predicting on unaligned vs aligned data INPUTS: pos_list: list of M position arrays, of size samples1, samples2, etc. data_list: list of M data arrays, each of size (N features) X (j samples). Used as the reference for the self-prediction, may be different than the aligned data with itself pos_list_aligned list of M position arrays, of size samples1, samples2, etc. for the aligned datasets (can be the same as pos_list) data_dict_aligned: element i is a list of the datasets from data_list aligned with dataset i. Each dataset must have N features, but can have different samples from the unaligned data For the other parameters, see predict_distance_CV OUTPUTS: unaligned_error_array: element (ij) is the error of predicting position in dataset j using predictor from i n_splitspmaxpredictor_namerTperiodicr)r%r'rcpredict_position_CVrer!r,&predict_position_from_predictor_object)rgr[rwrvrJr error_typerr&self_error_listself_predictor_listrr<rFpos_prederrorrt predictorunaligned_error_arrayr6r7aligned_error_arrayr r r get_cross_prediction_errorss8          rc& sd}d}d}t|} t| d} fddt| D} t|| |||||\} } ttj| dd}d}d |}||}tj||d||d \}}}g}td D]<\}}||krq||}tj |||d |d \}}| |q| t|t |}|| kr d }|d krt| D]\}}|| kr|d k|}|tj|dd}tjdd}%|d7}t| D]:}t|| dk}!||}"||}tj|"||%d |!dqP|%|q|dkr|S||fSdS)a; Use the prediction error to find the best data alignment, and normalize each session separately if indicated pos_list: list of position arrays for each space pca_dict_aligned_with_canonical: output from "normalize_pca_dict_aligned" function. Assumes it has the "canonical" space as well set_align_to: if not None, all sessions will be aligned to this one (rather than choosing the one with most SI) return_aligned_dict: if True, the dictionary with all the aligned PCAs (after normalization) and the chosen space is returned rrrrcsg|]}||qSr r rrr r rxrz*return_best_mCCA_space..rrrYrTrz Optimized alignment, error: %.2fr)decimals F projection3d)nrowsncolssqueezefigsizenum subplot_kwr)position_bin_size)axscattercbar)r)r)rN)r%r'rrbr!meanrcrr|rreargminr)aroundpltgcfnumbersubplotsr] compute_average_data_by_positionplot_pca_with_position set_axis_off tight_layoutfiguresubplot)&rgrrJplotreturn_error_listverbosecv_foldsrrr&sessions_to_test_alignmentrrr error_list canonical_ref pca_to_testrlrtrerror_list_to_avgr position_predrbest_alignmentpca_list_alignedrfig_numm_to_align_idx m_to_alignpca_list_to_plotfigaxsrrpos_binspca_avgrr rr return_best_mCCA_spaceas      (               r__main__rzTime Ellapsed: %.1f)rDrErFTTF)N)rD)rDrrr)rDFFT)(__doc__timescipynumpyr!numpy.mamamatplotlib.pyplotpyplotrmatplotlib.patchesrseabornsns scipy.linalgrproject_parametersr^rrrrrprocessing_functionsrcr objectr rxrrfrrrrAZTTr`seedr)showr r r r s8    F ] 5   D