πŸ”„ A realtime Database for JavaScript Applications

Overview
Announcement
             Version 10.0.0 is now released, read the ANNOUNCEMENT               

RxDB

A realtime Database for JavaScript Applications

RxDB (short for Reactive Database) is a NoSQL-database for JavaScript Applications like Websites, hybrid Apps, Electron-Apps, Progressive Web Apps and NodeJs. Reactive means that you can not only query the current state, but subscribe to all state changes like the result of a query or even a single field of a document. This is great for UI-based realtime applications in way that makes it easy to develop and also has great performance benefits. To replicate data between your clients and server, RxDB provides modules for realtime replication with any CouchDB compliant endpoint and also with custom GraphQL endpoints.

follow on Twitter


reactive.gif


Features
πŸ’» πŸ“± Multiplatform support for browsers, nodejs, electron, cordova, react-native and every other javascript-runtime
πŸ“¨ Reactive data-handling based on RxJS
🚣 Offline first let your app still work when the users has no internet
πŸ”„ Replication between client and server-data, compatible with pouchdbPouchDB, couchdbCouchDB and cloudantIBM Cloudant. There is also a plugin for a GraphQL replication
πŸ“„ Schema-based with the easy-to-learn standard of json-schema
🍊 Mango-Query exactly like you know from mongoDB and mongoose
πŸ” Encryption of single data-fields to protect your users data
πŸ“€ πŸ“₯ Import/Export of the database-state (json), awesome for coding with TDD
πŸ“‘ Multi-Window to synchronise data between different browser-windows or nodejs-processes
πŸ’… ORM-capabilities to easily handle data-code-relations and customize functions of documents and collections
πŸ”· Full TypeScript support for fast and secure coding (Requires Typescript v3.8 or higher)

Platform-support

RxDB is made so that you can use exactly the same code at

We optimized, double-checked and made boilerplates so you can directly start to use RxDB with frameworks like

Quickstart

Installation:

npm install rxdb --save

# peerDependencies
npm install rxjs --save

Import:

import { 
  addPouchPlugin,
  createRxDatabase,
  getRxStoragePouch
} from 'rxdb';

// add the pouchdb indexeddb adapter
addPouchPlugin(require('pouchdb-adapter-idb'));

// create a database
const db = await createRxDatabase({
    // the name of the database
    name: 'heroesdb',
    // use pouchdb with the indexeddb-adapter as storage engine.
    storage: getRxStoragePouch('idb'),
    // optional password, used to encrypt fields when defined in the schema
    password: 'myLongAndStupidPassword'
});

// create collections
await db.addCollections({
  heroes: {
    schema: mySchema
  }
});

// insert a document
db.heroes.insert({ name: 'Bob' });                          

Feature-Showroom (click to toggle)

Subscribe to query results

RxDB implements rxjs to make your data reactive. This makes it easy to always show the real-time database-state in the dom without manually re-submitting your queries.

db.heroes
  .find()
  .sort('name')
  .$ // <- returns observable of query
  .subscribe( docs => {
    myDomElement.innerHTML = docs
      .map(doc => '<li>' + doc.name + '</li>')
      .join();
  });

reactive.gif

MultiWindow/Tab

When two instances of RxDB use the same storage-engine, their state and action-stream will be broadcasted. This means with two browser-windows the change of window #1 will automatically affect window #2. This works completely offline.

multiwindow.gif

Replication

Because RxDB relies on glorious PouchDB, it is easy to replicate the data between devices and servers. And yes, the changeEvents are also synced.

sync.gif

EventReduce

One big benefit of having a realtime database is that big performance optimizations can be done when the database knows a query is observed and the updated results are needed continuously. RxDB internally uses the Event-Reduce algorithm. This makes sure that when you update/insert/remove documents, the query does not have to re-run over the whole database but the new results will be calculated from the events. This creates a huge performance-gain with zero cost.

Use-Case-Example

Imagine you have a very big collection with many user-documents. At your page you want to display a toplist with users which have the most points and are currently logged in. You create a query and subscribe to it.

const query = usersCollection.find().where('loggedIn').eq(true).sort('points');
query.$.subscribe(users => {
    document.querySelector('body').innerHTML = users
        .reduce((prev, cur) => prev + cur.username+ '<br/>', '');
});

As you may detect, the query can take very long time to run, because you have thousands of users in the collection. When a user now logs off, the whole query will re-run over the database which takes again very long.

anyUser.loggedIn = false;
await anyUser.save();

But not with the EventReduce. Now, when one user logs off, it will calculate the new results from the current results plus the RxChangeEvent. This often can be done in-memory without making IO-requests to the storage-engine. EventReduce not only works on subscribed queries, but also when you do multiple .exec()'s on the same query.

Schema

Schemas are defined via jsonschema and are used to describe your data.

const mySchema = {
    title: "hero schema",
    version: 0,                 // <- incremental version-number
    description: "describes a simple hero",
    primaryKey: 'name',         // <- 'name' is the primary key for the coollection, it must be unique, required and of the type string 
    type: "object",
    properties: {
        name: {
            type: "string"
        },
        secret: {
            type: "string",
        },
        skills: {
            type: "array",
            maxItems: 5,
            uniqueItems: true,
            item: {
                type: "object",
                properties: {
                    name: {
                        type: "string"
                    },
                    damage: {
                        type: "number"
                    }
                }
            }
        }
    },
    required: ["color"],
    encrypted: ["secret"] // <- this means that the value of this field is stored encrypted
};
Mango / Chained queries

RxDB can be queried by standard NoSQL mango queries, like you maybe know from other NoSQL Databases like mongoDB.

Also you can use the mquery api to create chained mango-queries.

// normal query
myCollection.find({
  selector: {
    name: {
      $ne: 'Alice'
    },
    age: {
      $gt: 67
    }
  },
  sort: [{ age: 'desc' }],
  limit: 10
})

// chained query
myCollection
  .find()
  .where('name').ne('Alice')
  .where('age').gt(18).lt(67)
  .limit(10)
  .sort('-age')
  .exec().then( docs => {
    console.dir(docs);
  });
Encryption

By setting a schema-field to encrypted, the value of this field will be stored in encryption-mode and can't be read without the password. Of course you can also encrypt nested objects. Example:

{
  "title": "my schema",
  "properties": {
    "secret": {
      "type": "string",
      "encrypted": true
    }
  },
  "encrypted": [
    "secret"
  ]
}
Level-adapters

The underlying pouchdb can use different adapters as storage engine. So you can use RxDB in different environments by just switching the adapter. For example you can use websql in the browser, localstorage in mobile-browsers and a leveldown-adapter in nodejs.

// this requires the indexeddb-adapter
RxDB.plugin(require('pouchdb-adapter-idb'));
// this creates a database with the indexeddb-adapter
const database = await RxDB.create({
    name: 'mydatabase',
    adapter: 'indexeddb' // the name of your adapter
});

There is a big ecosystem of adapters you can use.

Import / Export

RxDB lets you import and export the whole database or single collections into json-objects. This is helpful to trace bugs in your application or to move to a given state in your tests.

// export a single collection
const jsonCol = await myCollection.dump();

// export the whole database
const jsonDB = await myDatabase.dump();

// import the dump to the collection
await emptyCollection.importDump(json);


// import the dump to the database
await emptyDatabase.importDump(json);
Leader-Election

Imagine your website needs to get a piece of data from the server once every minute. To accomplish this task you create a websocket or pull-interval. If your user now opens the site in 5 tabs parallel, it will run the interval or create the socket 5 times. This is a waste of resources which can be solved by RxDB's LeaderElection.

myRxDatabase.waitForLeadership()
  .then(() => {
      // this will only run when the instance becomes leader.
      mySocket = createWebSocket();
  });

In this example the leader is marked with the crown β™›

reactive.gif

Key-Compression

Depending on which adapter and in which environment you use RxDB, client-side storage is limited in some way or the other. To save disc-space, RxDB uses a schema based keycompression to minimize the size of saved documents. This saves about 40% of used storage.

Example:

// when you save an object with big keys
await myCollection.insert({
  firstName: 'foo'
  lastName:  'bar'
  stupidLongKey: 5
});

// key compression will internally transform it to
{
  '|a': 'foo'
  '|b':  'bar'
  '|c': 5
}

// so instead of 46 chars, the compressed-version has only 28
// the compression works internally, so you can of course still access values via the original key.names and run normal queries.
console.log(myDoc.firstName);
// 'foo'

Getting started

Get started now by reading the docs or exploring the example-projects.

Contribute

Check out how you can contribute to this project.

Follow up

  • Follow RxDB on twitter to not miss the latest enhancements.
  • Join the chat on gitter for discussion.

Thank you

A big Thank you to every contributor of this project.

License

Apache-2.0

Issues
  • Unable to store blob type attachments in Electron

    Unable to store blob type attachments in Electron

    Case

    bug

    Issue

    I've got an error when trying to store an attachment in electron renderer process, with pouchdb adapter.

        file: Pick<File, 'name' | 'type'> & { blob: Blob }
    
        await putAttachment(
                  {
                    id: file.name,
                    data: file.blob,
                    type: file.type,
                  },
                  true
                );
    

    Throws:

    Uncaught (in promise) TypeError [ERR_INVALID_ARG_TYPE]: The first argument must be of type string 
    or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received an instance of Blob
    

    Versions

        "pouchdb-adapter-leveldb": "^7.2.2",
        "rxdb": "^9.12.1",
        "electron": "^11.1.1",
    

    Info

    • Environment: electron
    • Adapter: pouchdb
    • Stack: Typescript,React

    Code

    After I see the source code, problem in this method, Buffer.from can't accept a Blob parameter.

      createBlobBuffer: function createBlobBuffer(data, type) {
        var blobBuffer;
    
        if (_util__WEBPACK_IMPORTED_MODULE_3__.isElectronRenderer) {
          // if we are inside of electron-renderer, always use the node-buffer
          return Buffer.from(data, {
            type: type
          });
        }
      /* ... */
    

    And it is impossible to encrypt the blob type attachments, JSON.stringify also can't accept a Blob parameter.

    var _encryptValue = function _encryptValue(value) {
      return encrypt(JSON.stringify(value), this.password);
    };
    
    opened by Wuchv 35
  • RxDatabase.create(): Adapter not added. Use RxDB.plugin(require('pouchdb-adapter-[adaptername]');

    RxDatabase.create(): Adapter not added. Use RxDB.plugin(require('pouchdb-adapter-[adaptername]');

    Case

    🐞 BUG, but I don't know where 😭

    Issue

    RxDatabase.create(): Adapter not added. Use RxDB.plugin(require('pouchdb-adapter-[adaptername]');

    I'm trying to create a service which will handle all database stuff. When I'm using the database service I keep getting an error message telling me that IndexedDB adapter is not imported as PouchDB plugin. I have tried with WebSQL adapter and it works with a warning message telling me to use IndexedDB adapter because it will be deprecated soon.

    Info

    • Environment: Browser
    • Adapter: IndexedDB
    • Stack: Typescript, Angular

    Code

    import { Injectable } from '@angular/core';
    import { Logger } from '@nsalaun/ng-logger';
    
    import * as RxDB from 'rxdb';
    import * as PouchDBAdapterWebSQL from 'pouchdb-adapter-websql';
    import * as PouchDBAdapterIdb from 'pouchdb-adapter-idb';
    
    RxDB.plugin(PouchDBAdapterWebSQL);
    RxDB.plugin(PouchDBAdapterIdb);
    
    @Injectable()
    export class DatabaseService {
      private _database: RxDB.RxDatabase;
    
      constructor(private logger: Logger) {
        this.initDatabase();
      }
    
      private async initDatabase(): Promise<any> {
        this._database = await RxDB.create({
          name: 'seristdb',
          multiInstance: true,
          adapter: 'idb'
        });
    
        this._database.$.subscribe(event => this.logger.debug(event));
      }
    }
    

    Screenshots

    error

    iOS 
    opened by igorissen 32
  • ANNOUNCEMENT: The plans for the next major release

    ANNOUNCEMENT: The plans for the next major release

    Hello everyone. Atm I am planning the next major release where I want to change some breaking things.

    The list of changes is maintained here: https://github.com/pubkey/rxdb/blob/master/orga/before-next-major.md Notice that not everything in this list will be in the next major release, it depends on how much time I have.

    Please check the list and make pull requests with other changes you want to see. Also I would be good to see some opinions here and I will answer open questions of course.

    ANNOUNCEMENT :scream: 10.0.0 
    opened by pubkey 31
  • Unexpected token: keyword (function) UglifyJs

    Unexpected token: keyword (function) UglifyJs

    hi author, i trying to use rxdb with vuejs + cordova ( quasar framework ) but when build this bug happend.

    ERROR in js/0.bcc8ac658f5ce5687243.js from UglifyJs
    Unexpected token: keyword (function) [js/0.bcc8ac658f5ce5687243.js:590,6]
    

    more detail i posted here: https://github.com/quasarframework/quasar/issues/573 this error make app can not run on mobile cordova, please help !

    opened by lyquocnam 31
  • ANNOUNCEMENT: Please help testing 8.0.0-beta.9

    ANNOUNCEMENT: Please help testing 8.0.0-beta.9

    Hello everybody. Today I released [email protected] with improvements and changes.

    The focus of the new version lays on better defaults and better performance.

    Please help testing the new version by installing it via npm i [email protected] and report any bugs you may find.

    What has changed?:

    Every change in the current beta is still changeable and you are invited to discussion. If you have any other proposals for breaking changes, please speak now or hold your peace until the next major release.

    EDIT: Released 8.0.0-beta.2 EDIT: Released 8.0.0-beta.3 EDIT: Released 8.0.0-beta.4 EDIT: Released 8.0.0-beta.5 EDIT: Released 8.0.0-beta.6 EDIT: Released 8.0.0-beta.7 EDIT: Released 8.0.0-beta.8 EDIT: Released 8.0.0-beta.9

    help wanted ANNOUNCEMENT :scream: 8.0.0 
    opened by pubkey 28
  • RxDB automatic observe and change data in Vuex

    RxDB automatic observe and change data in Vuex

    Case

    RxDB automatic observe and change data in Vuex

    Issue

    1. Create new database, schemas, collections
    2. Insert some documents
    3. Get documents and store it into store (using Vuex)

    Error: Vuex throw error: [Vue warn]: Error in callback for watcher "function () { return this._data.$$state }": "Error: [vuex] Do not mutate vuex store state outside mutation handlers

    It looks like Rxdb always observe and automatically change the data so Vuex throw above error (conflict with Vuex)

    image

    More information: this issue didn't happen on other DB (PouchDB, Postgresql) Question: How to stop Observe in RxDB?

    Info

    • Environment: Electron
    • Adapter:LevelDB
    • Stack: VueJS, Vuex

    Code

    Line 20: store.dispatch('testplan/changeTreeViewData', result)

    import * as Database from '../../../backend/Database'
    import _ from 'lodash'
    import store from '../../store'
    
    async function getTestPlanTree () {
        var result
        const db = await Database.get();
        let categoriesCollection = await db['categories']
    	await categoriesCollection.find({
    	}).exec().then(async (categories) => {
            result = categories
            for(var i in categories){
                result[i].children = []
            }
            _.forEach(categories, async (category, index) => {
                result[index].children = await category.testsuites_
            })
        }).catch((err) => { console.log('error', err) })
        // console.log('result', result)
        store.dispatch('testplan/changeTreeViewData', result)
        return result
    }
    export default {
        getTestPlanTree
    }
    
    opened by tindecken 22
  • Subscriber breaks with react-native and async storage

    Subscriber breaks with react-native and async storage

    Case

    I was just developing an app using rxdb and react-native, when the time came to update a record on the database, suddenly a subscription on a collection broke

    Issue

    triggering an update on document (via atomicSet or any other update operation) breaks the query response on other parts of the app

    Reproducible example:

    I've created a repo with the issue:

    https://github.com/ospfranco/rxdb-rn-bug

    to reproduce:

    1. clone and run the app: yarn, yarn ios
    2. click on add random node a couple of times, so you have some elements
    3. click on one of the ids on the list
    4. click on randomly update selected node, it will do an atomic update and all of the sudden the list of elements will be cleared, and only the selected node will remain
    react-native 
    opened by ospfranco 21
  • feature suggestion .update function

    feature suggestion .update function

    Hello, I've been playing with this a bit, love it!

    The functionality I'm missing is to do updates in a mongo update(matcher, changes) manner. I've built a simple solution for that stealing some code from minimongo. Would you be willing to take a PR? Or is a lack of update a design decision?

    What I'm doing now is:

    
    users.find(matchingQuery).exec().then(function (docs) {
            docs.forEach(function (doc, index) {
              const newDoc = { ...doc._data }
              modify(newDoc, setQuery);
              delete newDoc._rev;
              delete newDoc._id;
              Object.keys(newDoc).forEach((el) => {
                doc[el] = newDoc[el];
              });
              doc.save()
            });
    });
    

    setQuery can be {$set: {}}, or use $rename, $pullAll $pop $push $unset etc. modify changes the newDoc according to the setQuery.

    I'm thinking of making the modify a standalone package, that would take a doc and return a new one, that could be a dependency in rxdb, and then I'd just add a simple update method. If not needed - not a problem! I will just polish it a bit for my own use :)

    FEATURE REQUEST :pray: 
    opened by lgandecki 19
  • Getting 'Unexpected end of JSON input' exception in encryption.js

    Getting 'Unexpected end of JSON input' exception in encryption.js

    Case

    bug

    Issue

    Getting the following exception:

    encryption.js:25 Uncaught (in promise) SyntaxError: Unexpected end of JSON input
        at Object.parse (<anonymous>)
        at Crypter._decryptValue (encryption.js:25)
        at crypter.js:62
        at Array.map (<anonymous>)
        at Crypter.decrypt (crypter.js:59)
        at RxCollection._handleFromPouch (rx-collection.js:192)
        at rx-collection.js:314
        at Array.map (<anonymous>)
        at rx-collection.js:313
    

    Not exactly sure which part of my code triggers it, since the stack trace references none of my files.

    Info

    • Environment: browser
    • Adapter: IndexedDB
    • Stack: React

    Code

    Here's my schema:

    const schema = {
      title: 'User profile schema',
      description: 'Database schema for the profile of a user',
      version: 0,
      type: 'object',
      properties: {
        peerID: {
          type: 'string',
          primary: true,
        },
        handle: {
          type: 'string',
          encrypted: true,
        },    
        name: {
          type: 'string',
          encrypted: true,
        },
        location: {
          type: 'string',
          encrypted: true,
        },
        about: {
          type: 'string',
          encrypted: true,
        },
        shortDescription: {
          type: 'string',
          encrypted: true,
        },
        nsfw: {
          type: 'boolean',
          encrypted: true,
        },
        vendor: {
          type: 'boolean',
          encrypted: true,
        },
        moderator: {
          type: 'boolean',
          encrypted: true,
        },
        // Will work this in later
        moderatorInfo: {
          type: ['object', 'null'],
          encrypted: true,
        },
        contactInfo: {
          type: ['object', 'null'],
          encrypted: true,
          properties: {
            website: {
              type: 'string',
            },
            email: {
              type: 'string',
            },
            phoneNumber: {
              type: 'string',
            },
            social: {
              type: 'array',
              uniqueItems: true,
              items: {
                type: 'object',
                properties: {
                  type: {
                    type: 'string'
                  },
                  username: {
                    type: 'string'
                  },
                  proof: {
                    type: 'string'
                  },
                }
              }
            }
          }
        },
        colors: {
          type: ['object', 'null'],
          encrypted: true,
          properties: {
            primary: {
              type: 'string',
            },
            secondary: {
              type: 'string',
            },
            text: {
              type: 'string',
            },
            highlight: {
              type: 'string',
            },
            highlightText: {
              type: 'string',
            },
          }
        },
        avatarHashes: {
          type: ['object', 'null'],
          encrypted: true,
          properties: {
            tiny: {
              type: 'string'
            },
            small: {
              type: 'string'
            },
            medium: {
              type: 'string'
            },
            large: {
              type: 'string'
            },
            original: {
              type: 'string'
            },
          }
        },    
      },
      required: ['peerID', 'name']
    };
    

    Here's a dump of my db:

    {
      "name": "a0e62fc96f1ca251998d512db1353c21f008d8c071b5c46a6212f9ab8e854c1c9",
      "instanceToken": "oodzoxksdi",
      "encrypted": true,
      "passwordHash": "a76bb4891fe30596c51e410f63e42bd9",
      "collections": [
        {
          "name": "profile",
          "schemaHash": "132ae4ac54c7552c18d7e47881deca12",
          "encrypted": true,
          "passwordHash": "a76bb4891fe30596c51e410f63e42bd9",
          "docs": [
            {
              "moderatorInfo": "U2FsdGVkX1+/wnlXNw6+M43WciG7cJLSWGAMFZswKiM=",
              "contactInfo": "U2FsdGVkX18vJeCWXulo+yzOH8LZvo/116TZz2G44xg=",
              "colors": "U2FsdGVkX19NDXFNWacnl6lhnIrvWZWVj1tCbxKyuik=",
              "avatarHashes": "U2FsdGVkX19H+VaarclSvJrfKQAENQ0UNM3N/zsmBIg=",
              "name": "U2FsdGVkX1/FV1R+rnIAokULhcZJLl+gkvcstrtgzgU=",
              "shortDescription": "U2FsdGVkX19oniIoqdi2wfrzfTtooaKoVhRGIWvTDRI=",
              "handle": "U2FsdGVkX1+ZlspmtKqfPLQiQ69poS6Yus5GE6MUHt4=",
              "location": "U2FsdGVkX19RpHIltTXvMcKJIX5mAJxiHgoMLbGA+sE=",
              "about": "U2FsdGVkX184Sc8xWQSI00mE/TwlPeiZSZi9/XxGw0o=",
              "nsfw": "U2FsdGVkX1+lfcIIxHwN6vvMHUiEPCMH33ux2frXpnI=",
              "vendor": "U2FsdGVkX1+nqr63Th0KW8qdgjzrZCTC+k+ytIdye4w=",
              "moderator": "U2FsdGVkX19qmzgLZJ2lO6b51u9C85Rt8VlXWlRB9oc=",
              "peerID": "QmRHvW41Ga6wQuayQNbrhwrTghasabxwWwqmT5hrWXQLoj"
            }
          ]
        }
      ]
    }
    
    opened by rmisio 17
  • two hotfixes

    two hotfixes

    there are what seem to me 2 bugs in the v4.0.2 which render RxDB unusable for me - this fixes this (so far):

    • first bug ... every top level property is 2 chars long after splitting it to "." ... so >=2 must be >2
    • 2nd bug ... on keycompression you try to only recurse on nested objs, but typeof null === "object"
    opened by joehannes-zz 16
  • use deleted flag for pushing in graphql replication

    use deleted flag for pushing in graphql replication

    This PR contains:

    • A BUGFIX

    Describe the problem you have without this PR

    Fixes #3436

    I don't feel like this is a full fix yet as I'm not sure how if a doc is deleted in pouch that gets set via the deleted flag. I think this has something to do with checking for if this is a delete operation, but I'm not sure why that was changed via https://github.com/pubkey/rxdb/commit/480c56bfa4d18d304525e1cd2b2e20b34a5364ce#diff-9d7498b685187264496834aaae5e2d04b1273c86b921c6b1bc1a2bf5c797f6d8 and I don't want to rebreak things without understanding the fix.

    Todos

    • [X] Tests
    • [ ] Documentation
    • [ ] Typings
    • [X] Changelog
    opened by chrisdrackett 1
  • `syncGraphQL` should optionallly allow credentials/cookies to be sent with push and pull requests

    `syncGraphQL` should optionallly allow credentials/cookies to be sent with push and pull requests

    Case

    I believe this may have just been an overlook option since I cannot find an open issue or PR related to it.

    When implementing GraphQL syncing, the pull and push requests to the graphql endpoint should optionally allow the users cookies to be included with the request. This is very helpful for authentication purposes, for example, when using an http-only cookie which stores a JWT.

    Issue

    There is currently no option on a collection's syncGraphQL method to include user credentials with push and/or pull requests.

    Code

    I see in the source code for the GraphQL sync plugin, that graphql-client is being used to make requests to the graphql API. This library accepts an option for including 'credentials'.

    GraphQL sync plugin code graphql-client option source code

    Proposed Solution

    It would be very easy to also accept an options field in a collections syncGraphQL method to include credentials which can then be funnelled to the underlying http graphql client. See link to GraphQL sync plugin code above.

    I am more than willing to submit a pull request if that would be acceptable. Please let me know.

    Thank you!

    opened by tcandens 1
  • deletedFlag not getting set on push during graphQL replication

    deletedFlag not getting set on push during graphQL replication

    Case

    Bug

    Info

    • RxDB Version: 10

    Code

    using a config like in the docs:

    const replicationState = myCollection.syncGraphQL({
        url: 'http://example.com/graphql', // url to the GraphQL endpoint
        push: {
            queryBuilder: pushQueryBuilder, // the queryBuilder from above
            batchSize: 5, // (optional) amount of documents that will pulled out of the storage at once. This does not affect how many documents are send to the server in a single request.
            modifier: d => d // (optional) modifies all pushed documents before they are send to the GraphQL endpoint. Returning null will skip the document.
        },
        deletedFlag: 'deleted', // the flag which indicates if a pulled document is deleted
        live: true // if this is true, rxdb will watch for ongoing changes and sync them
    });
    

    We're finding that we get an error like the following after updating to RxDB 10:

    [{"extensions": {"code": "BAD_USER_INPUT", "exception": [Object]}, "locations": [[Object]], "message": "Variable \"$calendar\" got invalid value { color: \"#D796F8\", createdAt: 1629398262001, id: \"4600CBB8-C57C-4A00-81F2-CBA8D89AA8F0\", title: \"My Calendar\", type: \"NORMAL\", updatedAt: 1633630617642 }; Field \"isDeleted\" of required type \"Boolean!\" was not provided."}]
    

    I'm looking at https://github.com/pubkey/rxdb/blob/master/src/plugins/replication-graphql/index.ts#L345 and wondering if this should be the deleted flag? seems like the deleted flag used to be used: https://github.com/pubkey/rxdb/blob/c4be6acf422db4d33d2aa394779f0368312fd81e/src/plugins/replication-graphql/index.ts#L294

    but I haven't had time to dig in and see what is actually going on yet.

    opened by chrisdrackett 2
  • [Feature Request] Change Middleware to have an input and output to make working with typescript easier

    [Feature Request] Change Middleware to have an input and output to make working with typescript easier

    Case

    Feature Request

    Issue

    If using typescript working with middleware is hard to impossible. This is because the middleware is currently designed to modify data in place, sometimes transforming it from an input type to an output type. This isn't something that typescript can handle as far as I can tell which leads a developer to have to pick a single type or opt-out of typescript in middleware all together.

    A suggestion would be to switch middleware to have an input and then return its output. This way you could add distinct input and output types and both ends could be verified via typescript.

    Info

    • RxDB Version: 10
    opened by chrisdrackett 0
  • Add unit test for #3417

    Add unit test for #3417

    This PR contains:

    Unit test

    Describe the problem you have without this PR

    #3417

    opened by swnf 1
  • Replication doc key

    Replication doc key

    fixes #3394

    This PR contains:

    • A NEW FEATURE

    Describe the problem you have without this PR

    As part of updating to 10.0 we noticed that we're getting events for the replication status when watching collection document updates. This is new and was breaking some things in our app. For now we're filtering this out by constructing the ID ourselves, but I thought others might have this same issue and it would be nice for RxGraphQLReplicationState to provide these id's to make it easer.

    The other thing this PR does is provide a reset function on RxGraphQLReplicationState. We need this because we currently have a way for a user to delete data on our server (but not in their local app) Right now we have the following bug:

    1. User chooses to delete server data
    2. User chooses to store data on server again
    3. User's data does not go to server because RxDB thinks all data has already been replicated.

    I figured it might be useful for others to be able to do this same thing.

    Todos

    • [X] Tests
    • [ ] Documentation
    • [ ] Typings
    • [ ] Changelog
    opened by chrisdrackett 4
  • Feature Request: Need a way to reset graphql replication

    Feature Request: Need a way to reset graphql replication

    Case

    Feature Request

    Issue

    I'd like to add a reset function on RxGraphQLReplicationState. We need this because we currently have a way for a user to delete data on our server (but not in their local app) Right now we have the following bug:

    1. User chooses to delete server data (we turn off replication and delete data on server)
    2. Later User chooses to store data on server again
    3. !! Users data does not go to server because RxDB thinks all data has already been replicated.

    This feature would have a way to remove the local documents that track replication so that replication will start again from scratch.

    opened by chrisdrackett 0
  • added failing test for collection name-reuse

    added failing test for collection name-reuse

    This PR contains:

    A new, failing test

    Describe the problem you have without this PR

    When I remove a collection and then recreate it (a new collection with the same name), operations (e.g. insert) on that collection result in unhandled rejection errors: Uncaught (in promise) Error: primary missing !!

    Operations don't directly throw, so I added an unhandledrejection eventListener in order to fail the test.

    Other (more difficult to directly "test") issues coincide with this error, and I'm hoping that fixing this issue will result in my other issues resolving. For example, before the recreation of a collection, a find observation on the collection is appropriately run/updated when an initial-replication completes, but this doesn't happen after the collection is recreated. (I'm not asking for help with this other issue at this point -- just giving additional info)

    opened by jfrancos 3
  • chore(deps): update dependency mocha to v9.1.3

    chore(deps): update dependency mocha to v9.1.3

    WhiteSource Renovate

    This PR contains the following updates:

    | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | mocha (source) | 9.0.2 -> 9.1.3 | age | adoption | passing | confidence | | @types/mocha | 8.2.3 -> 9.0.0 | age | adoption | passing | confidence |


    Release Notes

    mochajs/mocha

    v9.1.3

    Compare Source

    :bug: Fixes

    :nut_and_bolt: Other

    v9.1.2

    Compare Source

    :bug: Fixes

    :nut_and_bolt: Other

    v9.1.1

    Compare Source

    :bug: Fixes

    v9.1.0

    Compare Source

    :tada: Enhancements

    v9.0.3

    Compare Source

    :bug: Fixes


    Configuration

    πŸ“… Schedule: At any time (no schedule defined).

    🚦 Automerge: Disabled due to failing status checks.

    β™» Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

    πŸ”• Ignore: Close this PR and you won't be reminded about these updates again.


    • [ ] If you want to rebase/retry this PR, check this box.

    This PR has been generated by WhiteSource Renovate. View repository job log here.

    opened by renovate[bot] 0
  • chore(deps): update dependency typescript to v4.4.4

    chore(deps): update dependency typescript to v4.4.4

    WhiteSource Renovate

    This PR contains the following updates:

    | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | typescript (source) | 4.3.5 -> 4.4.4 | age | adoption | passing | confidence |


    Release Notes

    Microsoft/TypeScript

    v4.4.4

    Compare Source

    This patch release contains fixes for a performance regression in --build mode by only calling realpath on package.json files only when those files are known to exist on disk, and to only perform this work under --watch mode.


    For release notes, check out the release announcement.

    For the complete list of fixed issues, check out the

    Downloads are available on:

    v4.4.3

    Compare Source

    This patch release contains a fix for leaking file watchers on package.json, and a minor fix for completions in JavaScript files.


    For release notes, check out the release announcement.

    For the complete list of fixed issues, check out the

    Downloads are available on:

    v4.4.2

    Compare Source

    For release notes, check out the release announcement.

    For the complete list of fixed issues, check out the

    Downloads are available on:


    Configuration

    πŸ“… Schedule: At any time (no schedule defined).

    🚦 Automerge: Disabled due to failing status checks.

    β™» Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

    πŸ”• Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box.

    This PR has been generated by WhiteSource Renovate. View repository job log here.

    opened by renovate[bot] 0
Releases(10.2.0)
  • 10.2.0(Oct 13, 2021)

  • 10.1.0(Sep 27, 2021)

    Other:

    • Refactored the migration plugin for better performance by writing the documents in bulk requests
    • Added svelte example #3287 Thanks @bkeating
    • Improved error messages

    Bugfixes:

    • #3319 Graphql replication checkpoint was not deleted after running RxDatabase.remove()
    • Fixed spelling of recieved -> received everywhere. The old getters are still useable but deprecated #3392. Thanks chrisdrackett
    Source code(tar.gz)
    Source code(zip)
  • 10.0.3(Aug 9, 2021)

  • 10.0.0(Jul 20, 2021)

    Read the release announcement here

    Breaking:

    • Setting a primaryKey for a schema is now required.

    • When using the type RxJsonSchema<DocType> the DocType is now required.

    • A JsonSchema must have the required array at the top level and it must contain the primary key.

    • Outgoing data is now Readonly typed and deep-frozen in dev mode

    • RxDocument.putAttachment() no longer supports string as data, only Blob or Buffer.

    • Changed the default of putAttachment to skipIfSame=true.

    • Removed the deprecated atomicSet(), use atomicPatch() instead.

    • Removed the deprecated RxDatabase.collection() use RxDatabase().addCollections() instead.

    • Moved everything pouchdb related to the pouchdb plugin.

    • Pouchdb plugins are not longer added via addRxPlugin() but addPouchPlugin(). (RxDB plugins are still added via addRxPlugin).

    • Removed plugin hook preCreatePouchDb.

    • Removed the watch-for-changes plugin, this is now directly integrated into the pouchdb RxStorage.

    • Removed the adapter-check plugin. (The function adapterCheck is move to the pouchdb plugin).

    • Calling RxDatabase.server() now returns a promise that resolves when the server is started up.

    • Changed the defaults of PouchDBExpressServerOptions from the server() method, by default we now store logs in the tmp folder and the config is in memory.

    • Renamed replication-plugin to replication-couchdb to be more consistend in naming like with replication-graphql

      • Renamed RxCollection().sync() to RxCollection().syncCouchDB()
    • Renamed the functions of the json import/export plugin to be less confusing

      • dump() is now exportJSON()
      • importDump() is now importJSON()
    • RxCollection uses a separate pouchdb instance for local documents, so that they can persist during migrations.

    Features:

    • Added support for composite primary keys.

    Other:

    • Moved all should never happen errors into own error code.

    Typings:

    • Improved typings of error codes.
    Source code(tar.gz)
    Source code(zip)
  • 9.21.0(Jun 30, 2021)

  • 10.0.0-beta.4(Jun 25, 2021)

    Read the release announcement

    Breaking:

    • Setting a primaryKey for a schema is now required.

    • When using the type RxJsonSchema<DocType> the DocType is now required.

    • A JsonSchema must have the required array at the top level and it must contain the primary key.

    • Outgoing data is now Readonly typed and deep-frozen in dev mode

    • RxDocument.putAttachment() no longer supports string as data, only Blob or Buffer.

    • Changed the default of putAttachment to skipIfSame=true.

    • Removed the deprecated atomicSet(), use atomicPatch() instead.

    • Removed the deprecated RxDatabase.collection() use RxDatabase().addCollections() instead.

    • Moved everything pouchdb related to the pouchdb plugin.

    • Pouchdb plugins are not longer added via addRxPlugin() but addPouchPlugin(). (RxDB plugins are still added via addRxPlugin).

    • Removed plugin hook preCreatePouchDb.

    • Removed the watch-for-changes plugin, this is now directly integrated into the pouchdb RxStorage.

    • Removed the adapter-check plugin. (The function adapterCheck is move to the pouchdb plugin).

    • Calling RxDatabase.server() now returns a promise that resolves when the server is started up.

    • Changed the defaults of PouchDBExpressServerOptions from the server() method, by default we now store logs in the tmp folder and the config is in memory.

    • Renamed replication-plugin to replication-couchdb to be more consistend in naming like with replication-graphql

      • Renamed RxCollection().sync() to RxCollection().syncCouchDB()
    • Renamed the functions of the json import/export plugin to be less confusing

      • dump() is now exportJSON()
      • importDump() is now importJSON()
    • RxCollection uses a separate pouchdb instance for local documents, so that they can persist during migrations.

    Features:

    • Added support for composite primary keys.

    Other:

    • Moved all should never happen errors into own error code.

    Typings:

    • Improved typings of error codes.
    Source code(tar.gz)
    Source code(zip)
  • 10.0.0-beta.2(Jun 25, 2021)

    Read the release announcement

    Breaking:

    • Setting a primaryKey for a schema is now required.

    • When using the type RxJsonSchema<DocType> the DocType is now required.

    • A JsonSchema must have the required array at the top level and it must contain the primary key.

    • Outgoing data is now Readonly typed and deep-frozen in dev mode

    • RxDocument.putAttachment() no longer supports string as data, only Blob or Buffer.

    • Changed the default of putAttachment to skipIfSame=true.

    • Removed the deprecated atomicSet(), use atomicPatch() instead.

    • Removed the deprecated RxDatabase.collection() use RxDatabase().addCollections() instead.

    • Moved everything pouchdb related to the pouchdb plugin.

    • Pouchdb plugins are not longer added via addRxPlugin() but addPouchPlugin(). (RxDB plugins are still added via addRxPlugin).

    • Removed plugin hook preCreatePouchDb.

    • Removed the watch-for-changes plugin, this is now directly integrated into the pouchdb RxStorage.

    • Removed the adapter-check plugin. (The function adapterCheck is move to the pouchdb plugin).

    • Calling RxDatabase.server() now returns a promise that resolves when the server is started up.

    • Changed the defaults of PouchDBExpressServerOptions from the server() method, by default we now store logs in the tmp folder and the config is in memory.

    • Renamed replication-plugin to replication-couchdb to be more consistend in naming like with replication-graphql

      • Renamed RxCollection().sync() to RxCollection().syncCouchDB()
    • Renamed the functions of the json import/export plugin to be less confusing

      • dump() is now exportJSON()
      • importDump() is now importJSON()
    • RxCollection uses a separate pouchdb instance for local documents, so that they can persist during migrations.

    Features:

    • Added support for composite primary keys.

    Other:

    • Moved all should never happen errors into own error code.

    Typings:

    • Improved typings of error codes.
    Source code(tar.gz)
    Source code(zip)
  • 10.0.0-beta.1(Jun 22, 2021)

    Read the full release article

    Breaking:

    • Setting a primaryKey for a schema is now required.

    • When using the type RxJsonSchema<DocType> the DocType is now required.

    • Outgoing data is now Readonly typed and deep-frozen in dev mode

    • RxDocument.putAttachment() no longer supports string as data, only Blob or Buffer.

    • Changed the default of putAttachment to skipIfSame=true.

    • Removed the deprecated atomicSet(), use atomicPatch() instead.

    • Removed the deprecated RxDatabase.collection() use RxDatabase().addCollections() instead.

    • Moved everything pouchdb related to the pouchdb plugin.

    • Pouchdb plugins are not longer added via addRxPlugin() but addPouchPlugin(). (RxDB plugins are still added via addRxPlugin).

    • Removed plugin hook preCreatePouchDb.

    • Removed the watch-for-changes plugin, this is now directly integrated into the pouchdb RxStorage.

    • Removed the adapter-check plugin. (The function adapterCheck is move to the pouchdb plugin).

    • Calling RxDatabase.server() now returns a promise that resolves when the server is started up.

    • Changed the defaults of PouchDBExpressServerOptions from the server() method, by default we now store logs in the tmp folder and the config is in memory.

    • Renamed replication-plugin to replication-couchdb to be more consistend in naming like with replication-graphql

      • Renamed RxCollection().sync() to RxCollection().syncCouchDB()
    • Renamed the functions of the json import/export plugin to be less confusing

      • dump() is now exportJSON()
      • importDump() is now importJSON()
    • RxCollection uses a separate pouchdb instance for local documents, so that they can persist during migrations.

    Source code(tar.gz)
    Source code(zip)
  • 9.20.0(May 15, 2021)

    Bugfixes:

    • Auto-cancel one time couchdb replications to not cause a memory leak
    • Fixed another memory leak when calling the couchdb replication many times.

    Features:

    • Added the backup-plugin

    Other:

    • Updated rxjs to version 7.0.1
    Source code(tar.gz)
    Source code(zip)
  • 9.18.0(Apr 26, 2021)

  • 9.17.0(Apr 14, 2021)

  • 9.16.0(Apr 12, 2021)

    Features:

    • Added RxDatabase.migrationStates() which returns an observable to observe the state of all ongoing migrations.
    • Added startupPromise to the returned object of RxDatabase().server()

    Bugfixes:

    • Ensure every background task is done when RxDatabase.destroy() resolves. #2938

    Other:

    • Added analytics to docs page
    Source code(tar.gz)
    Source code(zip)
  • 9.15.0(Feb 24, 2021)

  • 9.14.0(Feb 14, 2021)

    Features:

    • Added RxReplicationState.awaitInitialReplication()

    Bugfixes:

    • Using the replication plugins must not required to also use leader-election
    • Refactor QueryCache.triggerCacheReplacement() to not spawn setTimeout regulary. This is needed for server side rendering with angular universal.

    Other:

    Source code(tar.gz)
    Source code(zip)
  • 9.13.0(Feb 10, 2021)

  • 9.12.1(Jan 24, 2021)

  • 9.12.0(Jan 3, 2021)

    Features:

    • Allow primary and ref at the same time in a schema. #2747

    Bugfixes:

    • #2705 when use bulkInsert to insert doc, the rxDocument property on changeEvent is an object, not a RxDocument instance. Thanks @qinyang912
    • When the mutation function of atomicUpdate() has thrown once, it was not possible to use it again.
    Source code(tar.gz)
    Source code(zip)
  • 9.11.0(Dec 13, 2020)

    Features:

    • Added putAttachment(skipIfSame), if set to true a write will be skipped if the attachment already exists with the same data.

    Bugfixes:

    • awaitInitialReplication() resolves on failed replication #2745. Thanks @dome4
    • insertLocal() not emitted the state change across tabs

    Other:

    • Added name identifier to RxPlugin
    • Throw error when dev-mode plugin is added multiple times because there is no way that this was done intentional likely the developer has mixed core and default usage of RxDB.
    • Fix reported security problem with the query builders mquery api.
    Source code(tar.gz)
    Source code(zip)
  • 9.10.1(Nov 23, 2020)

  • 9.10.0(Nov 23, 2020)

    Features:

    • Added RxCollection.getLocal$() and RxDatabase.getLocal$() to observe local documents.
    • Added RxDatabase.addCollections() to create multiple collections at once. Is faster and better typed than the now deprecated RxDatabase.collection()

    Other:

    • Improved typings for pouchdb.changes()
    • Moved from travisci to github actions
    Source code(tar.gz)
    Source code(zip)
  • 9.9.0(Nov 10, 2020)

    Other:

    • Improved startup performance by doing a index-exist check
    • Added check for properties to dev-mode schema check
    • Add better checks for query input in dev-mode
    Source code(tar.gz)
    Source code(zip)
  • 9.8.0(Nov 2, 2020)

    Features:

    Bugfixes:

    • (types) Returned values of syncGraphQL() did not type-match with RxGraphQLReplicationState
    • RxDocument.atomicUpdate() now does a retry on 409 write conflicts

    Other:

    • Added authentication to graphql example
    • Deprecated RxDocument.atomicSet(). Used atomicPatch instead, it works better with typescript
    • (docs) added workarounds for 6-connection limit at couchdb replication #2659. Thanks @MuresanSergiu
    Source code(tar.gz)
    Source code(zip)
  • 9.7.1(Oct 22, 2020)

  • 9.7.0(Oct 21, 2020)

    Bugfixes:

    • RxLocalDocument.$ emitted to often on changes #2471
    • Fix typings of RxReplicationState.collection

    Features:

    • Allow to skip docs in push/pull modifiers with the graphql-replication. #2552 Thanks @DDoerner
    • Allow to type the data of RxLocalDocument like myDatabase.getLocal<{foo: string}>('bar')

    Other:

    • Refactored GraphQL replication to run faster #2524 Thanks @corinv
    Source code(tar.gz)
    Source code(zip)
  • 9.6.0(Sep 7, 2020)

  • 9.5.0(Aug 2, 2020)

  • 9.4.0(Jul 23, 2020)

  • 9.3.0(Jun 26, 2020)

    Features:

    • Added (beta) RxCollection.findByIds() to get many documents by their id with a better performance.

    Other:

    • Added typings for pouch.allDocs()
    Source code(tar.gz)
    Source code(zip)
  • 9.2.0(Jun 21, 2020)

  • 9.1.0(May 31, 2020)

Owner
Daniel Meyer
git push origin hamster
Daniel Meyer
Realm is a mobile database: an alternative to SQLite & key-value stores

Realm is a mobile database that runs directly inside phones, tablets or wearables. This project hosts the JavaScript versions of Realm. Currently we s

Realm 4.6k Oct 19, 2021
πŸ‰ Reactive & asynchronous database for powerful React and React Native apps ⚑️

A reactive database framework Build powerful React and React Native apps that scale from hundreds to tens of thousands of records and remain fast ⚑️ W

Nozbe 7.8k Oct 21, 2021
Couchbase Lite binding for React Native Android & iOS

react-native-couchbase-lite NOTE: This plugin works for Couchbase Lite 1.x only. For new projects, we recommend to use the Couchbase Lite 2.0 Native A

Couchbase Labs 113 Jul 12, 2021
A AsyncStorage based mongoose like storage for react-native

React Native mongoose (remobile) A AsyncStorage based mongoose like storage for react-native Installation npm install @remobile/react-native-mongoose

YunJiang.Fang 21 Jul 24, 2021
React native Library for creating Local DB models

React Native DB Models This wrapper is built on top of React Native Store and provides a better and improved Database layer for asynchronous DB transa

Rishabh Mehan 173 Jun 12, 2021
A simple database base on react-native AsyncStorage.

react-native-store A simple database base on react-native AsyncStorage. Installation $ npm install react-native-store --save Upgrading from previous v

thewei 570 Oct 13, 2021
SQLite3 Native Plugin for React Native for iOS, Android, Windows and macOS.

SQLite3 Native Plugin for React Native for Android, iOS, Windows and macOS. This plugin provides a WebSQL-compatible API to store data in a react nati

Takuya Matsuyama 257 Oct 7, 2021
PouchDB adapter using ReactNative SQLite as its backing store

pouchdb-adapter-react-native-sqlite PouchDB adapter using ReactNative SQLite as its backing store. Why? SQLite storage performs much faster than Async

Takuya Matsuyama 102 Oct 2, 2021
Run async tasks that retry after a crash, connection loss or exception

react-native-persistent-job Run parametized asynchronous functions (called jobs) that will re-run in cases of failure, connection loss, application cr

Gabi Nir 80 Mar 22, 2020