Skip to content

feat(Atomic): Make read write the data atomic to avoid corruption of database #449

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Aug 1, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
fcea4bb
feat(Adapter): Add concept of adapter to read and write data
Belphemur Aug 1, 2022
dcb7ff0
tests(Adapter): Add proper testing for adapters
Belphemur Aug 1, 2022
8afd432
refactor(imports): cleanup imports
Belphemur Aug 1, 2022
817c581
fix(JsonAdapter): Don't override the data property
Belphemur Aug 1, 2022
c8f264b
feat(Adapter): Let the user decide what adapter to use if they want t…
Belphemur Aug 1, 2022
002a72a
feat(Configuration): Force giving a config object to the constructor
Belphemur Aug 1, 2022
4a51239
feat(AtomicFileAdapter): Add support for fsync
Belphemur Aug 1, 2022
764cdf4
feat(Async): Make the whole library async
Belphemur Aug 1, 2022
2b7d573
tests(Async): Update jsondb tests
Belphemur Aug 1, 2022
26355b0
fix(Array): Fix array not properly async
Belphemur Aug 1, 2022
dbb68d6
tests(Array): Update array tests
Belphemur Aug 1, 2022
13e51d9
fix(HumanReadable): Fix missing humanreadable
Belphemur Aug 1, 2022
84c3fec
tests(AsyncThrow): Fix test with ToThrow
Belphemur Aug 1, 2022
5b8e622
chore(mkdirp): Remove unneeded deps
Belphemur Aug 1, 2022
1bb5ec3
test(delete): fix deleting file
Belphemur Aug 1, 2022
9d43b01
tests(Fix): Fix all remaining tests
Belphemur Aug 1, 2022
4861726
docs(Async/Await): Add documentation about async await
Belphemur Aug 1, 2022
d4a55cf
tests(Import): Fix imports for tests
Belphemur Aug 1, 2022
15205da
chore(config): clean up the config contract
Belphemur Aug 1, 2022
5b1f000
test(adapter): add test for the configWithAdapter
Belphemur Aug 1, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 40 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[![NodeJs](https://github.com/Belphemur/node-json-db/actions/workflows/nodejs.yml/badge.svg)](https://github.com/Belphemur/node-json-db/actions/workflows/nodejs.yml)[![codecov](https://codecov.io/gh/Belphemur/node-json-db/branch/master/graph/badge.svg?token=J3Ppt4UCbY)](https://codecov.io/gh/Belphemur/node-json-db)[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2FBelphemur%2Fnode-json-db.svg?type=shield)](https://app.fossa.io/projects/git%2Bgithub.com%2FBelphemur%2Fnode-json-db?ref=badge_shield)
[![NodeJs](https://github.com/Belphemur/node-json-db/actions/workflows/nodejs.yml/badge.svg)](https://github.com/Belphemur/node-json-db/actions/workflows/nodejs.yml)[![codecov](https://codecov.io/gh/Belphemur/node-json-db/branch/master/graph/badge.svg?token=J3Ppt4UCbY)](https://codecov.io/gh/Belphemur/node-json-db)[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2FBelphemur%2Fnode-json-await db.svg?type=shield)](https://app.fossa.io/projects/git%2Bgithub.com%2FBelphemur%2Fnode-json-db?ref=badge_shield)

[![NPM](https://nodei.co/npm/node-json-db.png?downloads=true&stars=true)](https://nodei.co/npm/node-json-db/)
[![NPM](https://nodei.co/npm/node-json-await db.png?downloads=true&stars=true)](https://nodei.co/npm/node-json-db/)

> A simple "database" that use JSON file for Node.JS.

Expand Down Expand Up @@ -52,17 +52,17 @@ var db = new JsonDB(new Config("myDataBase", true, false, '/'));
// Pushing the data into the database
// With the wanted DataPath
// By default the push will override the old value
db.push("/test1","super test");
await db.push("/test1","super test");

// It also create automatically the hierarchy when pushing new data for a DataPath that doesn't exists
db.push("/test2/my/test",5);
await db.push("/test2/my/test",5);

// You can also push directly objects
db.push("/test3", {test:"test", json: {test:["test"]}});
await db.push("/test3", {test:"test", json: {test:["test"]}});

// If you don't want to override the data but to merge them
// The merge is recursive and work with Object and Array.
db.push("/test3", {
await db.push("/test3", {
new:"cool",
json: {
important : 5
Expand All @@ -85,35 +85,35 @@ This give you this results :

// You can't merge primitive.
// If you do this:
db.push("/test2/my/test/",10,false);
await db.push("/test2/my/test/",10,false);

// The data will be overriden

// Get the data from the root
var data = db.getData("/");
var data = await db.getData("/");

// From a particular DataPath
var data = db.getData("/test1");
var data = await db.getData("/test1");

// If you try to get some data from a DataPath that doesn't exists
// You'll get an Error
try {
var data = db.getData("/test1/test/dont/work");
var data = await db.getData("/test1/test/dont/work");
} catch(error) {
// The error will tell you where the DataPath stopped. In this case test1
// Since /test1/test does't exist.
console.error(error);
};

// Deleting data
db.delete("/test1");
await db.delete("/test1");

// Save the data (useful if you disable the saveOnPush)
db.save();
await db.save();

// In case you have a exterior change to the databse file and want to reload it
// use this method
db.reload();
await db.reload();

```

Expand All @@ -135,6 +135,12 @@ import { Config } from 'node-json-db/dist/lib/JsonDBConfig'

const db = new JsonDB(new Config("myDataBase", true, false, '/'));
```
#### v2.0.0

JsonDB is now using the concept of async/await for all its calls since we read from the database file on demands and
depending how the database is configured, we might write at each push.



#### Typing
With TypeScript, you have access to a new method: getObject<T> that will take care of typing your return object.
Expand All @@ -150,10 +156,10 @@ interface FooBar {
}
const object = {Hello: "World", World: 5} as FooBar;

db.push("/test", object);
await db.push("/test", object);

//Will be typed as FooBar in your IDE
const result = db.getObject<FooBar>("/test");
const result = await db.getObject<FooBar>("/test");
```


Expand All @@ -170,79 +176,79 @@ import { Config } from 'node-json-db/dist/lib/JsonDBConfig'
const db = new JsonDB(new Config("myDataBase", true, false, '/'));

// This will create an array 'myarray' with the object '{obj:'test'}' at index 0
db.push("/arraytest/myarray[0]", {
await db.push("/arraytest/myarray[0]", {
obj:'test'
}, true);

// You can retrieve a property of an object included in an array
// testString = 'test';
var testString = db.getData("/arraytest/myarray[0]/obj");
var testString = await db.getData("/arraytest/myarray[0]/obj");

// Doing this will delete the object stored at the index 0 of the array.
// Keep in mind this won't delete the array even if it's empty.
db.delete("/arraytest/myarray[0]");
await db.delete("/arraytest/myarray[0]");
```

#### Appending in Array
```javascript
// You can also easily append new item to an existing array
// This set the next index with {obj: 'test'}
db.push("/arraytest/myarray[]", {
await db.push("/arraytest/myarray[]", {
obj:'test'
}, true);


// The append feature can be used in conjuction with properties
// This will set the next index as an object {myTest: 'test'}
db.push("/arraytest/myarray[]/myTest", 'test', true);
await db.push("/arraytest/myarray[]/myTest", 'test', true);

```

#### Last Item in Array
```javascript
// Add basic array
db.push("/arraytest/lastItemArray", [1, 2, 3], true);
await db.push("/arraytest/lastItemArray", [1, 2, 3], true);

// You can easily get the last item of the array with the index -1
// This will return 3
db.getData("/arraytest/lastItemArray[-1]");
await db.getData("/arraytest/lastItemArray[-1]");


// You can delete the last item of an array with -1
// This will remove the integer "3" from the array
db.delete("/arraytest/lastItemArray[-1]");
await db.delete("/arraytest/lastItemArray[-1]");

// This will return 2 since 3 just got removed
db.getData("/arraytest/lastItemArray[-1]");
await db.getData("/arraytest/lastItemArray[-1]");
```
#### Count for Array
```javascript
//
db.push("/arraytest/list", [{id: 65464646155, name: "test"}], true);
await db.push("/arraytest/list", [{id: 65464646155, name: "test"}], true);

// You can have the number of element, in this case = 1
let numberOfElement = db.count("/arraytest/list");
let numberOfElement = await db.count("/arraytest/list");
```

#### Get Index in Array
```javascript

// You can have the current index of an object
db.push("/arraytest/myarray", {id: 65464646155, name: "test"}, true);
db.getIndex("/arraytest/myarray", 65464646155);
await db.push("/arraytest/myarray", {id: 65464646155, name: "test"}, true);
await db.getIndex("/arraytest/myarray", 65464646155);
// By default, the property is 'id'
// You can add another property instead
db.getIndex("/arraytest/myarray", "test", "name");
await db.getIndex("/arraytest/myarray", "test", "name");

// It's useful if you want to delete some object
db.delete("/arraytest/myarray[" + db.getIndex("/arraytest/myarray", 65464646155) + "]");
await db.delete("/arraytest/myarray[" + await db.getIndex("/arraytest/myarray", 65464646155) + "]");
```

#### Nesting in Array
```javascript
// You can easily access any nested array and their object
//You can also append to nested array other array
db.push("/arraytest/myarray",
await db.push("/arraytest/myarray",
[
[
{
Expand All @@ -262,7 +268,7 @@ db.push("/arraytest/myarray",

//This will return the first object (obj: 'test')

db.getData("/arraytest/myarray[0][0]");
await db.getData("/arraytest/myarray[0][0]");

```
### Exception/Error
Expand Down Expand Up @@ -294,7 +300,7 @@ db.getData("/arraytest/myarray[0][0]");
## Object with `separator` in key
Object pushed with key containing the `separator` character won't be reachable. See [#75](https://github.com/Belphemur/node-json-db/issues/75).

Please consider the `separator` as a reserved character by node-json-db.
Please consider the `separator` as a reserved character by node-json-await db.


# Contributors
Expand All @@ -321,4 +327,4 @@ db.getData("/arraytest/myarray[0][0]");


## License
[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2FBelphemur%2Fnode-json-db.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2FBelphemur%2Fnode-json-db?ref=badge_large)
[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2FBelphemur%2Fnode-json-await db.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2FBelphemur%2Fnode-json-db?ref=badge_large)
8 changes: 7 additions & 1 deletion jest.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,11 @@
module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
coverageProvider: "v8"
coverageProvider: "v8",
globals: {
"ts-jest": {
isolatedModules: true,
useESM: true
},
},
};
6 changes: 3 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
},
"homepage": "https://github.com/Belphemur/node-json-db",
"dependencies": {
"mkdirp": "~1.0.4"
"atomically": "^1.7.0"
},
"config": {
"commitizen": {
Expand Down Expand Up @@ -67,12 +67,12 @@
"@types/safe-regex": "^1.1.2",
"cz-conventional-changelog": "^3.3.0",
"husky": "^8.0.0",
"jest": "^27.2.0",
"jest": "^28.1.3",
"last-release-git": "^0.0.3",
"safe-regex": "~2.1.1",
"semantic-release": "^19.0.2",
"travis-deploy-once": "^5.0.11",
"ts-jest": "^27.0.5",
"ts-jest": "^28.0.7",
"typedoc": "^0.23.1",
"typescript": "^4.4.3",
"validate-commit-msg": "^2.14.0"
Expand Down
Loading