Transformers.jl

Transformers.jl is a Julia library that implements Transformer models for natural language processing tasks. Inspired by architectures like BERT, GPT, and T5, the library offers a modular and flexible interface for building, training, and using transformer-based deep learning models. It supports training from scratch and fine-tuning pretrained models, and integrates with Flux.jl for automatic differentiation and optimization.

Features

Implements standard Transformer architectures (BERT, GPT, etc.)
Modular design for custom model configuration
Pretraining and fine-tuning capabilities
Tokenization and positional encoding support
Compatible with Flux.jl and automatic differentiation
Support for GPU acceleration via CUDA.jl

Project Samples

Project Activity

See All Activity >

License

MIT License

Follow Transformers.jl

Transformers.jl Web Site

Other Useful Business Software

Simple, Secure Domain Registration

Get your domain at wholesale price. Cloudflare offers simple, secure registration with no markups, plus free DNS, CDN, and SSL integration.

Register or renew your domain and pay only what we pay. No markups, hidden fees, or surprise add-ons. Choose from over 400 TLDs (.com, .ai, .dev). Every domain is integrated with Cloudflare's industry-leading DNS, CDN, and free SSL to make your site faster and more secure. Simple, secure, at-cost domain registration.

Rate This Project

User Reviews

Be the first to post a review of Transformers.jl!

Additional Project Details

Operating Systems

Linux, Mac, Windows

Programming Language

Julia

Related Categories

Julia Natural Language Processing (NLP) Tool

Registered

2025-07-21

Similar Business Software

Haystack

Apply the latest NLP technology to your own data with the use of Haystack's pipeline architecture. Implement production-ready semantic search, question answering, summarization and document ranking for a wide range of NLP applications. Evaluate components and fine-tune models. Ask questions in...

See Software
BERT

BERT is a large language model and a method of pre-training language representations. Pre-training refers to how BERT is first trained on a large source of text, such as Wikipedia. You can then apply the training results to other Natural Language Processing (NLP) tasks, such as question...

See Software
InstructGPT

InstructGPT is an open-source framework for training language models to generate natural language instructions from visual input. It uses a generative pre-trained transformer (GPT) model and the state-of-the-art object detector, Mask R-CNN, to detect objects in images and generate natural...

See Software
Spark NLP

Experience the power of large language models like never before, unleashing the full potential of Natural Language Processing (NLP) with Spark NLP, the open source library that delivers scalable LLMs. The full code base is open under the Apache 2.0 license, including pre-trained models and...

See Software
GPT-4

GPT-4 (Generative Pre-trained Transformer 4) is a large-scale unsupervised language model, yet to be released by OpenAI. GPT-4 is the successor to GPT-3 and part of the GPT-n series of natural language processing models, and was trained on a dataset of 45TB of text to produce human-like text...

See Software
NLP Cloud

Fast and accurate AI models suited for production. Highly-available inference API leveraging the most advanced NVIDIA GPUs. We selected the best open-source natural language processing (NLP) models from the community and deployed them for you. Fine-tune your own models - including GPT-J - or...

See Software

Report inappropriate content

Transformers.jl

Julia Implementation of Transformer models

Get an email when there's a new version of Transformers.jl

Features

Project Samples

Project Activity

Categories

License

Follow Transformers.jl

User Reviews

Additional Project Details

Operating Systems

Programming Language

Related Categories

Registered