The new encryption package for React Native is ready.
✅ Re-built it using C++ JSI bindings
✅ Improved error handling
✅ Updated Android natives
✅ Supports hardware encryption
✅ Supports Biometrics
Give it a star! 😄
Imagine not having to deal with Cocoapods, Fastlane, Ruby, Gradle and actually having auto-complete, type-checking, and almost immediate errors when you are configuring your app
This is what a TypeScript build system could do
The final numbers are in
OP SQLite is not only 3x faster, it also consumes orders of magnitude less memory
The fastest SQLite module for React Native is born :)
I'm going to re-write as a normal JSI module
I will also use Swift 5.9 so I can use Swift functions directly from C++. Will be useful for other modules that want to migrate to 5.9
OP-SQLite is a success
It is 5x faster in iOS and 8x faster on Android as quick-sqlite
It uses 5x less memory
It supports DB encryption (via OP-SQLCipher)
All memory crashes from quick-sqlite are solved
Leave it a star and please consider sponsoring
I just noticed that since 0.73 Hermes is only 16 mbs, wireless run (via Wi-Fi) to iPhone now only takes some seconds to transfer App bundle + Hermes
On <0.72 it took even minutes because it seemed to copy the debug symbols
This is really awesome
Happy I got all these working
There are a few things I would like to tackle still: Bluetooth, a more advanced sqlite-mobx library, some sort of stand-alone module library that allows for easy C++/Swift/Kotlin mods
Nothing will work without sponsoring though
Are you tired of getting garbage stack traces on unhandled promise rejections? me too! I want to tackle this problem at the Hermes/RN level.
Give me a hand upvoting this and if you have some info, feel free to dump it here
@tmikov
Hacking quick-sqlite again
✅Queries are anywhere between 2x to 3x faster
✅Memory consumption is 50% to 80% smaller
✅It is now possible to run huge queries that before might have run out of memory
Here are a 300k records being loaded into memory
If you want to prevent the annoying metro message about recrawling when it starts. You can add this to your .watchmanconfig (for now, might get fixed upstream)
Once you understand shared pointers the world is your oyster
✅Memory consumption is halved
✅ Property access is a bit faster
~15ms iOS
~25ms Android
Queries are a tiny bit slower but for half the memory 🤝 A deal I'm willing to take
I was wondering where Expo makes money, looking at these prices, no need...
Curious what is the markup really, has anyone switched from raw Fastlane on GH or CircleCI to EAS and can dm me how much they ended up paying?
On GH with several weekly builds, we were at ~1-2k/month
Just discovered that SQLite supports Memory Mapped I/O and it's crazy. 300k records. Release Mode. Loading changes:
iOS: 510 ms → 450 ms (-60ms)
Android: 1125 ms → 1002 ms (-123ms)
Basically, for free, prepared statements don't even move the needle
New OP-SQLite release, fixes some crashes and errors
Please consider sponsoring, working on Sundays (as fun as it might be) is not really sustainable :)
Also: making Bento boxes is fun, is there any online generator?
I just fucked up my elbow bouldering, but here is the last results I got before leaving my house
old quick-sqlite: 8292ms
expo-sqlite: 4678ms
new quick-sqlite: 2840ms
300k rows of strings, ints, doubles
The best explanation I've seen of an ABI... all the C++ forums and posts are freaking impossible to understand. I wish I had this a couple of years ago.
I'm too dumb and I make mistakes, but that's also why I posted the code :)
Thanks
@jamesedmonston
Real performance of HostObject/JSI/ExpoModules... amazingly HostObject implementation is faster than a raw binding to the global context
It's nice to look at where things started by just playing around and then methodically replacing bits and bolts getting somewhere magical
iOS: 2719ms (quick-sqlite) → 634ms → 390ms
Android: 8851 (quick-sqlite) → 2220ms → 754ms 🤯
Taking a look inside Expo Modules code. It's a bunch of nice tricks. You are just creating classes that wrap up the obj-c callbacks and conversions for you.
No magic, just Swift-ObjC interop interacting with JSI C++, I wonder if they could simplify it with 5.9
Memory-mapped I/O + prepared statement SQLite is crazy
Here is a single insert/get compared to MMKV
SQLite is a powerhouse of a database and still comes within an inch of a key-value storage
Finally got some time to finish testing JSI/Expo Modules on Android. Here are both results for comparison
Android → Samsung s22 with debug compilation
iOS → iPhone 15 Pro with debug compilation
Finally got access to my S22 and could measure op-sqlite with the performance flag (and also rn-mmkv)
This is (as far as I can see) the end of the line for performance, an equivalent pure C++ program also executes in ~370±10ms
Until static hermes is here :)
Here is the current progress in converting turbo-secure-storage to a non-turbo-module
Speed compared to MMKV is okay, but by using the native APIs you get OS Hardware encryption/decryption and hardening of access
You also gain access to features like biometrics
Starting a project with Tauri. I see so many of the mistakes RN has already overcome. JSON serialization for native, untested alpha out-of-tree targets, missing docs
If companies knew/understood the great strides taken in the last years by RN they would not use anything else
It doesn't get any sweeter than that
Getting close to getting the first function call working, which actually unlocks a bunch of other interesting stuff :)
If you were using CodePush (or the Expo version) or reloading the JS bundle, quick-sqlite would crash. Just fixed this on OP-SQLite, so you can safely push and restart the JS Bundle
Dear Twitter followers, I'm free to tackle new projects. Especially C++ JSI (or Rust) projects
If you or your company need help, feel free to reach out!
Here is the first (small part) of implementing webcrypto/subtle on quick-crypto.
It unlocks pretty much all the imports/export jobs for all the algos, since the infrastructure is now there
It also paves the way for generating/signing/verifying
If you are using quick-sqlite, there is a major flaw with with TypedArrays (e.g. Uint8Array) and ArrayBuffers, data is not being persisted. Memory handling was botched, on app reload it tried to free non-allocated memory
This is all now fixed
Here is an almost direct JSI binding to Swift, no need to finagle with Obj-C/NS Classes.
Unfortunately, I don't think Swift plays well with Obj-C move semantics, but this is far easier than previous versions and probably faster too.
More info ↓🧵
I've created a new fork of op-sqlite, it uses SQLCipher which is a fork of SQLite that completely encrypts the database with a somewhat minimal overhead).
I need help testing it and making sure it is properly encrypted. Try it and let me know
Now that Flipper is going out the window, it seems there is no way to debug SQLite databases on the device...
Does anybody know of any alternative?
P.S. Android Studio db explorer doesn't work with op-sqlite because it doesn't use the Java API
When executing a SQLite query, most of the work is actually creating a plan on how to retrieve the data
You can avoid this cost by preparing the statement once and re-using it multiple times (with different params if necessary)
Coming to op-sqlite, does anybody want to sponsor?
My backpack got stolen the day after I arrived in Costa Rica. It had my computer, drone and passports inside, police won't do anything because I only have a rough (meters location)
I miss latam but when it sucks, it sucks big time
SQLite takes multitenancy to the next level. Just tested
@tursodatabase
's libsql-server hosting 100k databases, querying them all in a loop. It used 1.6GiB of RAM.
I could host a database for everyone in my town on a Raspberry Pi (and still have spare memory for
@The_Pi_Hole
).
I just had an idea for making op-sqlite reactive on a native level
This would be amazing and paired with mobx (or w/e u use) give you a real-time sqlite state management
However, it would be time-consuming as I would need to re-write a large chunk of the library
💰 anyone?
TIL you can get the local directory where the simulator emulates the file system simply by typing `po NSHomeDirectory` in the Xcode console when debugging/paused
In the mood for benchmarking stuff
Here is OP-S2 compared to SQLCipher and MMKV
Wanted to try expo-secure-store but the installation of expo kaputs itself with Yarn3, it should be on par with op-s2 expect a bit slower due to Expo Modules
New OP SQLite release
A lot of people have struggled with trying to open an existing database. I've added a new function to move one from assets into the file system easily
It also fixes a compilation issue with sqlcipher
I just updated op-sqlite example app to RN 0.73 and everything went fine, actually more than fine, if you have any issues let me know
Moving to Kotlin is **actually** really great, sorry for the confusion
Painless migration :)
If you are interesting in the complications of bridgeless you can follow the discussion of migrating op-sqlite here
Many thanks to the core RN team
Again this is a time consuming process, donate to further support
Final question I had, access time on objects
Unless I'm being too dumb with NativeState (seems only useful to be kept on the native side, no JS shenanigans)
To the Android devs, if your library depends on OpenSSL it will generate , but if another library also depends on it, both will generate it and build will fail
Is there any way to exclude only if libcrypto has already been imported
The only thing I miss about Flipper: SQLite database explorer
Not possible with Xcode/iOS
Android Studio only works when using the Java API
Was thinking of building something but it seems overly ambitious as a single contributor
I guess for now the best is an export function
Collected some repeated code in every project I work on. It will grow with time, for now it's some basic stuff.
Think lodash/underscore for everything else that pop-ups during normal app development.
TIL
auto blah = Blah()
auto shared_blah = std::make_shared<Blah>(blah)
!=
auto shared_blah = std::make_shared<Blah>();
shiet 🥲
There is a new op-sqlite release. It comes with prepared statements and memory leak fixes