Skip to content

Misc. bug: Intel container images keep getting No space left on device during CI Build #13052

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Thammachart opened this issue Apr 21, 2025 · 0 comments · Fixed by #13426
Closed

Comments

@Thammachart
Copy link
Contributor

Thammachart commented Apr 21, 2025

Name and Version

None
(This is on GitHub Actions service)

Operating systems

No response

Which llama.cpp modules do you know to be affected?

Documentation/Github

Command line

Problem description & steps to reproduce

This is from observation in container registry and GitHub Actions (Publish Docker image workflow)
https://github.com/ggml-org/llama.cpp/actions/workflows/docker.yml

one sample from workflow
https://github.com/ggml-org/llama.cpp/actions/runs/14567173390/job/40858195967

It show that since last month there's been reoccurring errors reported of
Unhandled exception. System.IO.IOException: No space left on device
Especially intel build, hence no recent images published

First Bad Commit

No response

Relevant log output

excerpted from GitHub Actions log:

#19 ERROR: mount callback failed on /tmp/containerd-mount1609910984: mount callback failed on /tmp/containerd-mount3948027268: failed to write compressed diff: failed to create diff tar stream: failed to copy: /tmp/containerd-mount3948027268/usr/local/lib/python3.10/dist-packages/numpy/core/_add_newdocs.py: write /var/lib/buildkit/runc-overlayfs/content/ingest/06e15eddbc06d10fad4a09540b027dd869b74c59e0052099baa0ad38a8c323b5/data: no space left on device

GitHub Actions internal error:

Unhandled exception. System.IO.IOException: No space left on device : '/home/runner/runners/2.323.0/_diag/Worker_20250421-041803-utc.log'
   at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.Diagnostics.TextWriterTraceListener.Flush()
   at System.Diagnostics.TraceSource.Flush()
   at GitHub.Runner.Common.Tracing.Dispose(Boolean disposing)
   at GitHub.Runner.Common.Tracing.Dispose()
   at GitHub.Runner.Common.TraceManager.Dispose(Boolean disposing)
   at GitHub.Runner.Common.TraceManager.Dispose()
   at GitHub.Runner.Common.HostContext.Dispose(Boolean disposing)
   at GitHub.Runner.Common.HostContext.Dispose()
   at GitHub.Runner.Worker.Program.Main(String[] args)
System.IO.IOException: No space left on device : '/home/runner/runners/2.323.0/_diag/Worker_20250421-041803-utc.log'
   at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.Diagnostics.TextWriterTraceListener.Flush()
   at GitHub.Runner.Common.HostTraceListener.WriteHeader(String source, TraceEventType eventType, Int32 id)
   at System.Diagnostics.TraceSource.TraceEvent(TraceEventType eventType, Int32 id, String message)
   at GitHub.Runner.Worker.Worker.RunAsync(String pipeIn, String pipeOut)
   at GitHub.Runner.Worker.Program.MainAsync(IHostContext context, String[] args)
System.IO.IOException: No space left on device : '/home/runner/runners/2.323.0/_diag/Worker_20250421-041803-utc.log'
   at System.IO.RandomAccess.WriteAtOffset(SafeFileHandle handle, ReadOnlySpan`1 buffer, Int64 fileOffset)
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.Diagnostics.TextWriterTraceListener.Flush()
   at GitHub.Runner.Common.HostTraceListener.WriteHeader(String source, TraceEventType eventType, Int32 id)
   at System.Diagnostics.TraceSource.TraceEvent(TraceEventType eventType, Int32 id, String message)
   at GitHub.Runner.Common.Tracing.Error(Exception exception)
   at GitHub.Runner.Worker.Program.MainAsync(IHostContext context, String[] args)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant