Adding inequality/segregation indices

[cdagnino]

Hi,

Do you think adding ways to calculate inequality/segregation would fit the scope of the quantecon project?
I already have one in this repository that I could contribute.

[jstac]

@cdagnino My instinct is yes, and the code looks well written. thanks for offering. My hesitation is that this code is code is quite isolated from what we have so far. If it was part of a broader set of code on inequality and segregation measures that would certainly be better.

@mmcky ?

[mmcky]

@cdagnino I am keen to diversify the library to include a distributed set of tools that are useful to economists. I will put some time this week into thinking how best we could organise this material, as I have been interested in assembling a datalab submodule that packages data related routines together.

My main question is how best to organise the code to sensibly accomodate a variety of different functions that are available from research that may compute the same (or similar) type of output through different methodologies. It would also be good to lay the foundation to allow for a collection of tools for computing inequality and segregation measures (as @jstac suggests). @cdagnino do you use other metrics in your work that would be useful to have in a library?

[cdagnino]

Hi,

@jstac @mmcky I agree 100% with the isolation problem of the code I linked.
If y'all have some thoughts on the interface/syntax for this datalab submodule (which I'm guessing would share some ideas from the more general Quantecon module) then I can adapt my code to give a first taste of inequality/segregation measures.

I think there are a bunch of inequality and segregation indices that could be interesting to people. Besides the one I coded, a Gini index, a Generalized entropy measure (with Between + Within decompositions) and Duncan Index might be interesting and could be easily made compatible with pandas dataframes. See this documents for some reference on inequality indices

The "point" estimates should be fine and relatively easy to code. Adding standard errors could be interesting, but more difficult and maybe outside the scope of this package.

[jstac]

I think these indices are a great example of what we could usefully code up for economists. And if there's a number of them we can consider making a subpackage.

@cdagnino Might you be willing to take the lead in developing this subpackage, if we decide to go ahead? We can probably devote RA resources to helping out, in that case...

[cdagnino]

@jstac I'd be interested, but I promised myself not to undertake anything semi-big until I finish my dissertation. I'd be happy to contribute with code and ideas though.

[jstac]

@cdagnino Might you be willing to contribute just a gini index calculator and a Lorenz curve calculator in the short term? Hopefully it wouldn't take long.

[cdagnino]

@jstac Yep! That sounds like doable, at least a simple version without standard errors.

Any suggestions on the implementation, particularly regarding how it should interact to the rest of the QuantEcon codebase?

I'm guessing I could write a Lorenz and Gini class that takes a numpy vector or pandas Series. Then these classes can be used to provide numbers for the indices, decompositions and basic plotting.

[jstac]

@cdagnino

Actually I needed them for a workshop so i just implemented them:

https://github.com/QuantEcon/Copenhagen_workshop_2018/blob/master/lecture_4/wealth_dynamics.ipynb

I use jit compilation based on Numba and, in the case of the Gini coefficient, automated multithreading via @njit with the parallel option.

So these are super fast.

It might make sense to build on top of these, adding the ability to accept pandas data frames or interact with plotting commands. (although we try to keep actual plotting commands out of the library, it's fine to have functions that, say, act on an ax object created through matplotlib.)

The functions I've written also need tests. I'd be very pleased to work together to improve these functions and then add them to the library.

@mmcky Just keeping you in the loop.

[cdagnino]

@jstac Took a first try here

Haven't implemented much, but I figured out the issue you had with the Pareto curve sampling and the Gini coefficient.
Turns out numpy doesn't have a real Pareto distribution, but rather a Pareto II or Lomax!
Once you "rescale", your graph works fine.

Stay tuned.

[jstac]

@cdagnino Nice, thanks! I guess you could use the same idea for the gini test --- at those extreme distributions the coefficient should be close to zero or close to one.

And thanks for the fix vis-a-vis the Pareto distribution!

If you get a chance to finish the tests then please submit this code as a PR and @mmcky (the package maintainer) will help get it merged.

We're just about to launch this site:

http://notes.quantecon.org/

and your notebook could be posted there to show people how to use the functions, if you're willing.

[jstac]

@rebekahanne Might you have time to help get this merged?

[rebekahanne]

@jstac Of course, but I don't think I have write access on this repository.

[jstac]

@rebekahanne Great, thanks. I'd appreciate it. The code is there already and we want to implement this functionality just as functions with corresponding tests. This is the main QuantEcon Python code library and @mmcky is the current gatekeeper. You'll need to fork the repository, collect together the code and tests in a branch and them submit a pull request. If you browse the repository you'll see how functions are combined with tests. @natashawatkins or @mmcky or @QBatista could help if you have questions.

[rebekahanne]

@jstac All right -- will do!