Make BaseOutput dataclasses picklable
#5234
Conversation
cbensimon
changed the title
BaseOutput dataclasses picklable
| # Don't call self.__setattr__ to avoid recursion errors | ||
| super().__setattr__(key, value) | ||
|
|
||
| def __reduce__(self): |
There was a problem hiding this comment.
That works! In Python, the __reduce__ function is more or less exclusively reserved for pickle no? So there should be no other side-effects that could be trigger here?
patrickvonplaten
left a comment
patrickvonplaten
left a comment
There was a problem hiding this comment.
Ok for me! I don't think Transformer's model output class has this reduce feature either (see here: https://github.com/huggingface/transformers/blob/391177441b133645c02181b57370ab12f71b88c4/src/transformers/utils/generic.py#L288) but I don't see a reason why we should not add it
|
cc @LysandreJik for visibility in case it's needed for Transformers |
Good catch, I wasn't aware that transformers were also using this dataclass OrderedDict class. |
Original PR from diffusers : huggingface/diffusers#5234
|
Just opened a draft PR in transformers : huggingface/transformers#26493 |
* Make BaseOutput dataclasses picklable * make style * Test * Empty commit * Simpler and safer
* Make `ModelOutput` serializable Original PR from diffusers : huggingface/diffusers#5234 * Black
* Make BaseOutput dataclasses picklable * make style * Test * Empty commit * Simpler and safer
* Make BaseOutput dataclasses picklable * make style * Test * Empty commit * Simpler and safer
Unpickling pipeline outputs currently produces an error (because the output is both a
@dataclassand anOrderedDict)This PR fixes this by setting a custom
__reduce__method on theBaseOutputclass