OPDB is a Rust-based project that scrapes data from the One Piece Wiki to create a structured and accessible data repository for the One Piece universe. It provides an API to access detailed information about devil fruits, pirate crews, ships, and more.
Whether you're a fan, developer, or researcher, OPDB makes it easy to explore the world of One Piece programmatically.
- Rust - The core programming language for the project.
- Tokio - For asynchronous runtime.
- Scraper - For HTML parsing and web scraping.
- Serde - For data serialization and deserialization.
- Devil Fruits: Detailed information about Paramecia, Zoan, and Logia types, including subtypes like Ancient and Mythical Zoan.
- Pirate Crews: Data about pirate crews, their captains, and affiliated ships.
- Ships: Information about ships, including their status, affiliation, and images.
- Non-Canon Detection: Identify whether a pirate crew, ship, or devil fruit is canon or non-canon.
- JSON Output: Export scraped data in JSON format for easy integration with other tools.
-
Clone the Repository:
git clone https://github.com/muhrifqii/opdb-rust.git cd opdb-rust
-
Install Rust: Ensure you have Rust installed. If not, install it via rustup:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
-
Build the Project:
cargo build --release
-
Run the Scraper:
cargo run --release -- --output <output_directory>
Replace
<output_directory>
with the path where you want the scraped data to be saved. If not specified, the default directory isdata
. -
Access the Data: The scraped data will be saved as JSON files in the specified output directory. For example:
df_type_infos.json
: Information about devil fruit types.df_list.json
: List of all devil fruits.pirates.json
: Details about pirate crews.ships.json
: Information about ships.
- API Server: Add a RESTful API to serve the scraped data dynamically.
- Enhanced Scraping: Support for additional One Piece data, such as characters, locations, and battles.
- GraphQL Support: Provide a GraphQL API for more flexible queries.
- Web Interface: Build a web-based UI for exploring the data visually.
- Data Validation: Improve data accuracy by cross-referencing multiple sources.
Contributions are welcome! If you have any ideas or suggestions, please open an issue or submit a pull request.
- Fork the repository.
- Create a new branch for your feature or bugfix.
- Submit a pull request with a clear description of your changes.
This project is licensed under the MIT License - see the LICENSE file for details.
Data is scraped from One Piece Wiki and is licensed under CC BY-SA 4.0.