This is a big commit - It should have been done on several
This is a big commit - It should have been done on several branches: 1) I've started work on a FileSystem implementation, which invovled fgactoring out some common code. 2) This was partly superceded...
View ArticleRefactored so I can add a binary key generator, for more compact
Refactored so I can add a binary key generator, for more compact keys. tests run but thats not saying all that much.
View ArticleUse tar instead of zip: zip keeps the table of contents in memory,
Use tar instead of zip: zip keeps the table of contents in memory, which means the export could run out of memory.
View ArticleUse tar instead of zip - writig tar files doesn't use memory,
Use tar instead of zip - writig tar files doesn't use memory, while writing zip files keeps a table of contents in memory, which is bad, under the circumstances.
View ArticleThis is really just an in progress dump: I have a binary key binding that...
This is really just an in progress dump: I have a binary key binding that might work, as long as you don't use blobs
View ArticleFixed various issues with the binary binding. This worked a little while ago...
Fixed various issues with the binary binding. This worked a little while ago when I had a separate map for blobs. Then I realised that meant I'd lost locality for blobs, so I merged them back into...
View ArticleBlobs with locality and binary keys now works. Haven't tested
Blobs with locality and binary keys now works. Haven't tested String based keys yet. Blob locality seems to improve write performance by 10-20% in my ad hoc test of importing the encyclopedia britanica.
View ArticleThis arranges for no-longer used blobs to be deleted. It also replaces the...
This arranges for no-longer used blobs to be deleted. It also replaces the export with one that can cope with the internal blob store or an external one - either way, the blobs are included in the...
View ArticleOptimisation which sorts changes by their key before submitting them to BDB.
Optimisation which sorts changes by their key before submitting them to BDB. BDB may do a fairly good job of this already, so its just an experiment.
View ArticleDon't delete the blobs of the new value after you have just written them! Add...
Don't delete the blobs of the new value after you have just written them! Add and modify no longer have enough in common to share a method. Separated them.
View ArticleI was actually quite wrong to think the stringKey order was the same as the...
I was actually quite wrong to think the stringKey order was the same as the binaryKey order - the numbers assigned to strings have a completely different ordering to the strings themselves. Its not...
View ArticleRe-applying an optimisation that avoids large byte array allocations and...
Re-applying an optimisation that avoids large byte array allocations and copies in blobs.
View ArticleThat solves the dynamic dispatch problem. I don't understand why the previous...
That solves the dynamic dispatch problem. I don't understand why the previous versiondidn't work, but this one does.
View ArticleDetect duplicate blobs. Count references to them, and delete them
Detect duplicate blobs. Count references to them, and delete them when they are no longer referenced.
View ArticleThis is a fast checksum of Paul Crowleys devising. I've added it
This is a fast checksum of Paul Crowleys devising. I've added it because SHA1DIgest was slowing things down significantly. I'll have to get him to add some notes about it somewhen.
View ArticleTheres some refactoring here: moving the key generators into
Theres some refactoring here: moving the key generators into separate top level classes. Support for DataStore is added to the persistence manager, and the blob store gets just enough file system...
View ArticleMissing from previous commit - skeleton of a data store
Missing from previous commit - skeleton of a data store
View ArticleAdd the ability to run tools which don't modify the database in
Add the ability to run tools which don't modify the database in read only mode, hence this gives us a hot backup. Dealt with a bug that causes database values to be padded with zeros to block...
View Article