Add DeiT Model

[Sohaib-Ahmed21]

Add VIT based DeiT (data-efficient image transformers) model to keras-hub along with its backbone, layers, tests and checkpoint conversion.
Paper: https://arxiv.org/pdf/2012.12877
Model card: https://huggingface.co/facebook/deit-base-distilled-patch16-384

Resources related to this PR:

• DeiT Checkpoint Conversion and Numerics Verification Demo (across multiple backends): Notebook Link
• DeiT End-to-End Demo (zero-shot and finetuning): Notebook Link
• Here are the converted DeiT presets from Hugging Face checkpoints for reference.

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[Sohaib-Ahmed21]

@divyashreepathihalli @JyotinderSingh , kindly approve the workflows.

[sachinprasadhs]

@Sohaib-Ahmed21 , Thanks for the PR, could you please add the checkpoint_conversion , and link a colab notebook here verifying the numerics, parameters matching and demonstrating end to end usage example of the model.

[Sohaib-Ahmed21]

@Sohaib-Ahmed21 , Thanks for the PR, could you please add the checkpoint_conversion , and link a colab notebook here verifying the numerics, parameters matching and demonstrating end to end usage example of the model.

Yeah sure, I'll do that and update the PR, thanks!

[Sohaib-Ahmed21]

Added the checkpoint_conversion, will share the parameter/numerics verification and end to end demo notebooks soon.

[Sohaib-Ahmed21]

I’m sharing the following resources related to this PR:

• DeiT Checkpoint Conversion and Numerics Verification Demo (across multiple backends): Notebook Link
• DeiT End-to-End Demo (zero-shot and finetuning): Notebook Link
• Here are the converted DeiT presets from Hugging Face checkpoints for reference.

[Sohaib-Ahmed21]

This PR is ready for review. Kindly approve the workflows and review the PR, thanks!

[sachinprasadhs]

Thanks for the demo notebook, added few more comments.

Also, in the colab can you assert the parameters matching between Keras and HF model.
Example, tiny variant has "params": 5524800, can you show the output where it matches with the HF params?

[Sohaib-Ahmed21]

Thanks for the detailed review. I'll address the reviews soon.

can you show the output where it matches with the HF params?

Yes, I'll show that.

[mattdangerw]

@Sohaib-Ahmed21 no the jax-gpu segfault popped up recently, it's probably related to our test environment (we haven't tracked it down yet). You can ignore.

[mattdangerw]

Few misc comments, but this is looking good!

[Sohaib-Ahmed21]

Failing test seems unrelated to PR. Kindly review, thanks!

[JyotinderSingh]

I’m sharing the following resources related to this PR:

• DeiT Checkpoint Conversion and Numerics Verification Demo (across multiple backends): Notebook Link
• DeiT End-to-End Demo (zero-shot and finetuning): Notebook Link
• Here are the converted DeiT presets from Hugging Face checkpoints for reference.

Please add these links to the PR description to make them easier to find.

[Sohaib-Ahmed21]

This PR seems ready for merge. Kindly review/merge it. Thanks!

[divyashreepathihalli]

In the demo colab

• why are you using the converter outside of the task class?
• this should be included in the task preprocessor right?

[sachinprasadhs]

I was looking into your generated files, especially in preprocessor.json, how are you populating norm_mean and norm_std

[Sohaib-Ahmed21]

In the demo colab

• why are you using the converter outside of the task class?
• this should be included in the task preprocessor right?

Other models also define the converter separately and link it to the preprocessor (e.g., ViT in keras-hub uses converter outside the task class here)

[Sohaib-Ahmed21]

I was looking into your generated files, especially in preprocessor.json, how are you populating norm_mean and norm_std

I have updated the presets at the link w.r.t new changes, now norm_mean and norm_std are not in preprocessor.json.

[Sohaib-Ahmed21]

@divyashreepathihalli @sachinprasadhs @mattdangerw This PR seems ready for merge. Kindly review/merge it. Thanks!

[Sohaib-Ahmed21]

Kindly review/merge this pull request, thanks!