Skip to content

riboseinc/isogit-lfs

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

87 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Aspirationally, a set of helpers to simplify working with Git LFS through Isomorphic Git.

Currently the API is fairly low-level and verbose, which could be improved.

However, touching working directory is out of scope. Callers are free to integrate the library into workflows based on bare Git repositories or repositories with working directories depending on their specifics.

Note
While Isomorphic Git maintenance status is unclear, this library is in turn not likely to receive new features.

Installation

Peer dependencies:

  • Isomorphic Git ^1.7.8

  • @aws-crypto/sha256-universal ^2.0.0

Usage

Example of using the library to read from LFS:

import { pointsToLFS } from '@riboseinc/isogit-lfs/util.js';
import { readPointer, downloadBlobFromPointer } from '@riboseinc/isogit-lfs';

// We need to know the remote URL of this repository
// in order to request LFS data
// (this relies on Git config just for the sake of example)
const remoteURL = await git.getConfig({ fs, gitdir, path: 'remote.origin.url' });

/**
 * Read data for an object in given repository (`gitdir`)
 * at particular commit (`oid`)
 * at particular repository-relative path (`path`)
 * using supplied filesystem implementation (`fs`).
 *
 * Different from `readBlob()` in that it also handles
 * LFS, downloading & caching objects automatically.
 */
async function readObject(fs, gitdir, oid, path): Promise<Uint8Array> {

  // 1) Read some blob from the repo
  const gitObject = await git.readBlob({ fs, gitdir, oid, filepath: path });

  // 2) Check if this blob points to LFS
  if (pointsToLFS(gitObject.blob)) {

    // If yes, 3) deserialize pointer
    const pointer = readPointer({
      gitdir,
      content: gitObject.blob,
    });

    // 4) Download or retrieve from cache
    return await downloadBlobFromPointer({
      fs,
      url: remoteURL,
      http,
    }, pointer);

  } else {
    // If not, just return the blob straight away
    return gitObject.blob;
  }

}

API

As of 0.2.0, API offers the following functions (where blobs are Uint8Array instances you obtain from or give to Isomorphic Git):

  • downloadBlobFromPointer({ http, headers, url, auth }, lfsPointer) ⇒ Promise<Uint8Array>

    where http is an HttpClient as supported by Isomorphic Git, URL is repository URL and lfsPointer is an object returned by readPointer().

    Downloads the object if necessary and returns the full blob.

    Uses LFS cache (which is hidden in Git repo structure, under .git/lfs for non-bare repositories) if the object had been previously retrieved.

  • uploadBlob({ http, headers, url, auth }, blob) ⇒ Promise<PointerInfo>

    where first argument is the same as for downloadBlobFromPointer(), and returned pointer info can be used to write a pointer file in place of actual object in Git repository (pass it through formatPointerInfo()).

  • readPointer({ gitdir, content }) ⇒ Pointer

    where gitdir behavior mimics that of Isomorphic Git (for non-bare repositories it’s not working directory but the .git in it) and content is a blob, Uint8Array-encoded version of a pointer you would find stored in Git instead of a full blob where LFS is used.

    Returns decoded LFS pointer structure together with Git repo object path to the full blob in cache (which at this point may or may not exist).

    Return value intended to be passed as the second argument to downloadBlobFromPointer().

  • util.pointsToLFS({ gitdir, content }) ⇒ boolean

    where the arguments are identical to readPointer() above.

    Returns true if given object’s blob that you obtained via Isomorphic Git is an LFS pointer in place of actual object data.

  • populateCache(gitdir, ref?) ⇒ Promise<void>

    where gitdir is same as for readPointer() and ref should probably be left at the default "HEAD".

    Attempts to download all LFS-stored objects in the tree of ref. Does not return anything or write anything in working directory but lets the blobs be cached for subsequently faster invocations of downloadBlobFromPointer().

    Note
    This particular function is a very tentative piece of API. Downloads are not particularly optimized, it’s likely quite slow.

Lower-level functions you probably won’t need to call:

  • readPointerInfo(blob) ⇒ PointerInfo

    decodes a Uint8Array blob of an LFS pointer into an LFS pointer structure sufficient for requesting actual LFS blob.

  • formatPointerInfo(lfsPointerInfo) ⇒ Uint8Array

    serializes LFS pointer structure to an appropriately formatted string and encodes it as a blob suitable to be stored in Git repository in place of actual object data.

Known shortcomings

  • The @aws-crypto/sha256-universal dependency is suboptimal. It pulls extra dependencies of its own, while it’s not that difficult to provide corresponding implementation using subtle crypto which is compatible between Node and modern browsers.

  • Originally written for Node runtime, but should work in browser as of 0.2.0.

  • Lacks automated tests.

Considered within scope

  • Implement batch uploads and downloads (parallelise requests? use native batch API?)

  • Find a way to generalize UA header handling

  • Make it work in browser runtime as well (if feasible?) — should be done in 0.2.0

About

LFS helpers on top of Isomorphic Git

Resources

License

Stars

Watchers

Forks

Packages

No packages published