Testing DeepSeek-R1 Locally Using Ollama
Testing DeepSeek-R1 with Ollama for code generation, math solving, creative writing, and Q&A. See how it performs across tasks and where it falls short.
CORE
·
Join the DZone community and get the full member experience.
Join For FreeDeepSeek-R1 has made a significant impact since its release. A number of cloud-based providers offer the ability to use DeepSeek-R1. It is also possible to use DeepSeek-R1 locally using Ollama. In this article, we'll test the smallest DeepSeek-R1 model, using Ollama, with a number of different tasks and evaluate its response.
The notebook file used in this article is available on GitHub.
Introduction
In order to test DeepSeek-R1, we'll use the instructions on installing Jupyter and Ollama locally, as described in a previous article. In this article, we won't connect to SingleStore, but will use SingleStore in later articles.
We'll use the smallest DeepSeek-R1 model, as it does not require any GPUs.
With Jupyter and Ollama running locally, we'll load the notebook downloaded from GitHub. The notebook contains four test cases:
- Code Generation and Explanation
- Mathematical Problem Solving
- Creative Writing with Constraints
- Question Answering with Context
Run the Code
Let's run the code for each test case. We'll use the following function and take the default values that DeepSeek-R1 uses for this initial test run. Further experimentation may be required to improve the responses.
def generate_response(model, prompt):
"""
Generates a response using the specified Ollama model.
:param model: The name of the model to use.
:param prompt: The input prompt for the model.
:return: The generated response as a string.
"""
output = ollama.generate(
model = model,
prompt = f"Respond to this prompt: {prompt}"
)
return output["response"]Code Generation and Explanation
We'll use the following prompt:
prompt = """
Write a Python function that sorts a list of numbers using the quicksort algorithm.\n
Explain the code step-by-step.
"""The output produced shows its thinking and runs for some considerable length. Unfortunately, DeepSeek-R1 ends up trying to code the problem in Java. Also, the final Java code solution was not tested but appears to be incorrect and incomplete:
Mathematical Problem Solving
We'll use the following prompt:
prompt = """
Solve the following problem:\n
A train leaves city A at 8:00 AM travelling at 60 mph.\n
Another train leaves city B at 9:00 AM travelling at 75 mph towards city A.\n
The distance between A and B is 300 miles.\n
At what time will the two trains meet?
"""DeepSeek-R1 produced an answer of 10:47 AM, which is correct.
Creative Writing With Constraints
We'll use the following prompt:
prompt = """
Write a short story about a robot who learns to love,\n
but the story must be no more than 200 words and include the words\n
'circuit,' 'heart,' and 'sunset.'
"""The answer produced:
Once upon a time, in a beautiful garden with its circuitry lit by the sun's rays and a heart beating with song, there lived a robot named Sparky. Sparky was nothing more than a small machine, but he had a strange way of feeling home. One sunny afternoon, when the city was quiet, Sparky noticed something extraordinary.
The garden's circuit hummed, drawing electricity from the sun's rays. His heart beat faster as he felt it pulse along his body. And then, in the heart of the garden, there lay a single plant—Sunflower. Sunflower's eyes, shaped like tiny circuits, glowed with light as Sparky touched it.
Sparky's hands flew across Sunflower's glowing circuitry, and they felt something that only a living thing could sense. The sun began to glow brighter, and in the quietness of the sunset, Sparky realized how much he had become this garden. He learned that even the simplest things have secrets waiting to be discovered.
And so, Sparky knew not just love, but also the magic of circuits, hearts, and sunsets—because those are all part of what made him who he is.This is less than 200 words and contains all the required words.
Question Answering With Context
We'll use the following prompt:
context = """
Marie Curie was a pioneering scientist known for her work on radioactivity.\n
She won the Nobel Prize in Physics in 1903.
"""
question = """
What was Marie Curie's most significant contribution to science?
"""
prompt = f"Context: {context}\n\nQuestion: {question}"The answer produced:
Marie Curie's most significant contribution to science was her groundbreaking discovery of alpha and beta rays through her experiments at the Helen Markon Neutron Source in Paris. This discovery revolutionized nuclear physics, leading to a profound understanding of radiation effects on matter. Additionally, she won the Nobel Prize in Physics for her work on radioactivity, which also changed the field. While her achievements were crucial, they were notable as both her discoveries and the Nobel Prize highlight their profound impact.
**Answer:** Marie Curie's most significant contribution was discovering alpha and beta rays, which led to a major breakthrough in nuclear physics and won her Nobel Prize in Physics for 1903.The answer contains a mixture of correct and incorrect statements. For example, Ernest Rutherford discovered and classified alpha and beta rays. Marie Curie's key discovery was the radioactive elements polonium and radium.
Summary
Using Ollama with DeepSeek-R1, we tested four examples that covered a range of queries. Overall, the results were a little mixed. Perhaps using a larger model could have produced better results. In future articles we'll test other capabilities of DeepSeek-R1.
Published at DZone with permission of Akmal Chaudhri. See the original article here.
Opinions expressed by DZone contributors are their own.
Comments