Marc Jourdan, Gizem Yüce, Nicolas Flammarion
ICML 2025
Recent advances in language modeling have underscored the role of preference feedback in enhancing model performance. This paper investigates the conditions under which preference feedback improves parameter estimation in classes of continuous parametric distributions. In our framework, the learner observes pairs of samples from an unknown distribution along with their relative preferences depending on the same unknown parameter. We show that preferences-based M-estimators achieve a better asymptotic variance than sample-only M-estimators, further improved by deterministic preferences. Leveraging the hard constraints revealed by deterministic preferences, we propose an estimator achieving an estimation error scaling of
Install Julia
wget https://julialang-s3.julialang.org/bin/linux/x64/1.11/julia-1.11.3-linux-x86_64.tar.gz
tar zxvf julia-1.11.3-linux-x86_64.tar.gz
export PATH="$PATH:/path/to/<Julia directory>/bin"
Install packages
julia
using Pkg;
Pkg.add(["JLD2", "Printf", "JSON", "Dates", "IterTools", "Distributed", "JuMP", "Ipopt", "HiGHS"]);
Pkg.add(["Random", "LinearAlgebra", "Distributions", "CPUTime"]);
Pkg.add(["StatsPlots", "ArgParse", "Statistics", "StatsBase", "Plots"]);
Pkg.add(["Pickle", "ColorSchemes", "Distributed", "LaTeXStrings"]);
using ArgParse, JLD2, Printf, JSON, Dates, IterTools, Random, CPUTime, Pickle;
using LinearAlgebra, Statistics, Distributions, StatsBase, LaTeXStrings;
using Plots, StatsPlots, ColorSchemes;
using JuMP, Ipopt, HiGHS;
using Distributed;
After installing Julia, to run the experiments presented in the paper, you can either use the custom commands (defined below) to only perform a given experiment or you can directly run the script.sh. Some experiments are computationally costly, hence we recommend to use more cores than solely four (-p4).
cd path_to_folder/code
mkdir experiments
mkdir data
To run our script with four cores:
chmod +x script.sh
bash script.sh 4
Note that the script doesn't include the plotting functions. Therefore, additional runs of the corresponding plotting functionalities have to be made afterwards (see below). As it is, you might need to be careful in the order in which you call the visualization functions since the plots are created for all the folders in the experiments folder, which don't have .pdf inside them. Also, better visualization can be obtained by commenting some lines of codes and replacing them with others.
cd path_to_folder/code
julia -O3 -p4 gaussian1d.jl --expe "error1d" --instance "Gaussian" --seed 42 --Nruns 1000 --Nsteps 10000 --batch 100 --sizemax 100
julia viz_gaussian1d.jl --format "svg"
cd path_to_folder/code
julia -O3 -p4 gaussianlarged.jl --expe "errorlarged" --instance "Gaussian" --dimension 20 --seed 42 --Nruns 100 --Nsteps 10000 --batch 400 --sizemax 25
julia viz_gaussianlarged.jl --format "svg"
cd path_to_folder/code
julia -O3 -p4 gaussianvard.jl --expe "errorvard" --instance "Gaussian" --dimMax 100 --dimStep 10 --seed 42 --Nruns 1000 --Nsteps 10000
julia viz_gaussianvard.jl --format "svg"
cd path_to_folder/code
julia -O3 -p4 gapsD.jl --expe "Matrices" --instance "Gaussian" --dimMax 100 --seed 42 --Nruns 1000000
julia viz_gapsD.jl --format "svg"
cd path_to_folder/code
julia -O3 -p4 laplace.jl --expe "error1d" --instance "Laplace" --seed 42 --Nruns 10 --Nsteps 10000 --batch 100 --sizemax 100
julia viz_laplace.jl --format "svg"
cd path_to_folder/code
julia -O3 -p4 rayleigh.jl --expe "error1d" --instance "Rayleigh" --seed 42 --Nruns 100 --Nsteps 10000 --batch 100 --sizemax 100
julia viz_rayleigh.jl --format "svg"
cd path_to_folder/code
julia -O3 -p4 betterEstim.jl --expe "betterEstim" --instance "Gaussian" --seed 42 --Nruns 100 --Nsteps 10000 --batch 100 --sizemax 100
julia viz_betterEstim.jl --format "svg"
cd path_to_folder/code
julia -O3 -p4 betterEstimLargeD.jl --expe "betterEstimlarged" --instance "Gaussian" --seed 42 --dimension 20 --seed 42 --Nruns 100 --Nsteps 10000 --batch 400 --sizemax 25
julia viz_betterEstimLarged.jl --format "svg"
cd path_to_folder/code
julia -O3 -p4 betterEstimvard.jl --expe "betterEstimVard" --instance "Gaussian" --dimMax 100 --dimStep 10 --seed 42 --Nruns 100 --Nsteps 10000
julia viz_BetterEstimVard.jl --format "svg"
cd path_to_folder/code
julia -O3 -p4 losses.jl --expe "losses" --instance "Gaussian" --seed 42 --Nruns 10 --Nsteps 10000 --batch 100 --sizemax 100
julia viz_losses.jl --format "svg"
cd path_to_folder/code
julia -O3 -p4 norm_and_regu.jl --expe "normregu" --instance "Gaussian" --seed 42 --Nruns 100 --Nsteps 10000 --batch 100 --sizemax 100
julia viz_normregu.jl --format "svg"
If you find this work useful in your own research, please consider citing it:
@article{jourdan2025learningwithpreference,
title={Learning Parametric Distributions from Samples and Preferences},
author={Jourdan, Marc and Y{\"u}ce, Gizem and Flammarion, Nicolas},
journal={International Conference on Machine Learning (ICML)},
year={2025}
}This codebase is released under MIT License.