Dataframe Extraction Utilities #32
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added `dataframe` method to `vaxflux.uptake.SeasonalUptakeModel` to create a pandas DataFrame of each posterior incidence time series.
Preliminary usage of the `SeasonalUptakeModel.dataframe` method and attempts to diagnose fitting issues.
TimothyWillard
force-pushed
the
dataframe-extraction-utilities
branch
from
11077f9 to
c855223
Compare
Added internal `vaxflux.covariates._covariate_categories_product` utility for quickly creating a product of user provided `CovariateCategories`s to loop over.
Added a function to generate a set of valid observations from user provided inputs for the purpose of generating test data.
Added a new demo notebook, `synthetic_data.ipynb` to demo the usage of `vaxflux.data.sample_dataset`.
TimothyWillard
force-pushed
the
dataframe-extraction-utilities
branch
from
2ba9ff1 to
fb127e4
Compare
Changed the `epsilon` argument of `vaxflux.data.sample_dataset` to not have a default value.
Fixed bug where `vaxflux.dates._infer_ranges_from_observations` could not create `SeasonRange` because the provided columns were called 'season_start/end_date' instead of 'start/end_date'. Fixed issue by adding helper to `vaxflux._util._rename_keys` to rename those dict keys before creating `SeasonRange` instance.
Attempted to fit a model to the synthetic data generated in the `synthetic_data.ipynb` demo notebook. However, model does not want to sample due to an indexing issue.
Fixed bug where the season and covariate category limited dims were incorrectly swapped for `vaxflux.covariates.GaussianRandomWalkCovariate`. Bug allowed for the case of those being equal to work, but would constraint seasons to the covariate category limited size when season was larger silently.
Successfully ran sampling after the season and category limited dims switch bug fix, but sampling still displays convergence issues.
Added broad keyword arguments to the `vaxflux.uptake.SeasonalUptakeModel` starting with 'constrain_prevalence', a boolean to dictate if a potential term constraining prevalence to [0,1] should be included.
Still getting fitting issues even with the prevalence constraint potential removed.
Allow a user to specify the observational sigma to relax fitting strictness. Appears to be the main cause of chain non-mixing from before.
Updated this demo with fits, don't look great but are mixing.
Added a custom option to pool the `epsilon` parameter across time series to share more information. This is helpful if there is good reason to believe variation will be similar (i.e. comparable population sizes), but won't work in general. Might be worth at some point trying to scale this parameter by population size.
Fits seem to display some odd upward bias in fits that need to be examined. First checking that analytically the sum is being done correctly and then seeing if the sampling procedure might be introducing bias.
For the synthetic data demo: * Shortened the data by a month since later data wasn't doing much, and * Added more plots for the parameter estimates, seems clear that the model is systematically mis-estimating many of the parameters.
TimothyWillard
force-pushed
the
dataframe-extraction-utilities
branch
from
90af5a2 to
8ff838b
Compare
Even with tightened priors the model still has biased estimates for some of the parameters.
Added `vaxflux.interventions` module with the `Intervention` class to represent the affect of an intervention on the parameters for uptake.
Added a pydantic object to represent the implementation of an intervention, allows for the ability to re-use interventions across seasons and covariates.
Attempted to implement a custom log-probability for the sum of gamma distributed variables, however the prelim fitting results did not look very promising.
This was referenced Apr 4, 2025
TimothyWillard
added a commit
that referenced
this pull request
Apr 15, 2025
Dataframe Extraction Utilities
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.