joafip sources

feedback / send your comments
mail to luc peuvrier

chat channel
joafip users mailing list
forums

BACK TO INDEX

RETOUR A L'INDEX

to get sources / distribution:

SCM Repositories

download from sourceforge.net

Maven depot

change log:

30 november 2012 : release 4.0.0b9

joafip-4.0.0b9-bin-dist md5 sha1

Sources md5 sha1

major newness:

foreground grabage sweep can be use even if garbage management disabled

bugs corrected:

red black tree management changed for deletion problems, impact FileTreeMap (joafip-pmap project)

know bugs:

background garbage management does not works

3 june 2012 : release 4.0.0b8

joafip-4.0.0b8-bin-dist md5 sha1

Sources md5 sha1

svn repository: https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-4.0.0b8

major newness:

project devided in sub projects and projects mavenized. projects are: joafip,joafip-common,joafip-rbtree,joafip-heapfile,joafip-file,joafip-collection,joafip-log4j,joafip-asm,joafip-java-agent,joafip-callback, joafip-jpa, joafip-example-database,joafip-example-hugemap,joafip-example-simplestore joafip-4test, joafip-testsuite
improved speed in saving and reading
use of FilePersistenceBuilder instead of "new FilePersistence(...)"
added file management parameters

name of files for storage
delete strategy when no more data stored
delete and open retry count

auto save for exclusive data access session:

have a virtual memory
quota setup changed
maintained in memory setup changed
auto save setup added to properties setup file
garbage management adapted

new proxy instance facilities, collections have newInstance method
declaration of class that no more exists
log of object too big to be serialized (StackOverflowError at serialization)
asm 3.3 embedded
java agent changed
import error reading xml value corrected
export speed up

1H26 for 6,001,103 (1,000,000 items) disk model Model ST3400620AS SATA 3Gb/s 400GB 7200 RPM
(same time for import)

kept in memory for object referenced by static field
persisted class byte code checked for persistence constraints

check field access constraints, direct field access disallow to force use of getter and/or setter
check private method invocation of other instance
check package visibility method interception
can setup the prohibited package by security manager, this information is used by proxy class generator

static field persistence feature removed because a non sense
heap file project make able to create a map in file with a comparable key
new joafip-pmap project, a file persisted tree map
heap data manager implemented in btree+ (not use by joafip since HeapFileDataManager speeder)
improve speed for file access using mapped byte buffer

minor changes:

added search last in red black tree manager
red black tree can now be set to manage unique or duplicate values
concurrent modification optional for tree based collection,
concurrent modification disabled by default, can be enabled
speed up when concurrent modification disabled
added modification listener for:

PLinkedList,PTreeList,ArrayList,PLinkedTreeSet,PLinkedHashSet,PTreeSet,PHashSet,PHashMap, PlinkedHashMap, PTreeMap, PLinkedTreeMap

PArrayList implement Deque
PLinkedTreeSet and PLinkedHashSet implement List
setSubstitutionOfJavaUtilCollection can be enabled or disabled, enabled by default
removeSubstitution added in IFilePersistence
new java agent implementation (performance to be improved)

bugs corrected:

search in red black tree manager did not return the first element of group of duplicate values
<T> T[] toArray(final T[] t) implementation corrections
conversion definition of TreeListSupport not defined
after test add, iterator and descending iterator correction for list and tree
after test add, sublist correction
conversion management correction
in garbage management for stored using serialization
stored enum management corrected in case of storage file erase
failure of write byte while serializing
too much warn failed to read root record #1, try to reduce, now warn only once at file persistence opening
tree sub set remove and contains correction

29 october 2010 : release 3.1.0b1

joafip-jar-3.1.0b1 md5 sha1

Sources md5 sha1

svn repository: https://joafip.svn.sourceforge.net/svnroot/joafip/branches/joafip-3.1.0branch/tags/joafip-3.1.0b1

concurrent modification while iterate correction for collections
PArrayList iterator set next do not update iterator next element

03 may 2010

joafip-jar-3.1.0

Sources md5 sha1

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-3.1.0

conversion on the fly and conversion at import changed
file cache optimization : speed up
memory leak free checked
save of objects detached to root optimized
PLinkedTreeSet and PLinkedHashSet implements Deque
new proxy instance creation correction

24 march 2010

joafip-jar-3.0.1b2

Sources md5 sha1

https://joafip.svn.sourceforge.net/svnroot/joafip/branches/joafip-3.0.1branch/tags/joafip-3.0.1b2

no lazy load of serializable did not works corrected

28 february 2010

joafip-3.0.1b1 md5 sha1
joafip-java-agent-3.0.1b1 md5 sha1

Sources md5 sha1

https://joafip.svn.sourceforge.net/svnroot/joafip/branches/joafip-3.0.1branch/tags/joafip-3.0.1b1

bug corrected:

had failures with PArrayList#add(int,Object)
problems with multiple instance with same value for class wrapping primitive type (eg java.lang.Integer)
mutable enum storage failure

enhancement

immutable and mutable enum storage management
IFilePersistence#storedEnum deprecated,
use IFilePersistence#storedImmutableEnum or IFilePersistence#storedMutableEnum
better session checking for proxy

7 february 2010

joafip-3.0.1 md5 sha1
joafip-java-agent-3.0.1 md5 sha1

Sources md5 sha1

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-3.0.1

changes

AloneDataAccessSession deprecated and became ExclusiveDataAccessSession

enhancement

ExclusiveDataAccessSession and associated feature fully tested
instrumentation.properties have now "on" option
object i/o management refactoring
set maximum object maintained in memory ( not unload ) after a save
save static field of class, good for singleton storage
conversion "on the fly" and for import change
manage no more existing class (no more existing to be declared in conversion definition file)
static or transient modifier added to field export

bugs corrected

java agent fully changed, new approach
added properties for creation and setup using properties
management of field list in conversion definition

17 December 2009

joafip-3.0.0rc1 md5 sha1
joafip-java-agent-3.0.0rc1 md5 sha1

Sources md5 sha1

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-3.0.0rc1

enhancements:

profiled to be faster, really faster
major change: data model identifier management
conversion on the fly
@DeprecatedInStoreClass annotation for old objects
conversion definition for import and conversion on the fly
can clear persisted data when file persistence is opened
- out of opened data access session
- at end of data access session
save without closing data access session ( AloneDataAccessSession )
creation of new proxy instance
@StorableClass annotation and flag set to allow or not class not annotated @StorableClass
can define old class fields list and declare field removed in conversion definition
collection comparator for list IEqualsHashComparator
added alone data access session that enable object saving without closing and able application to keep reference on saved object
setting of substitution class of a class. object substitution is saved and substituted restored using
a substitute object manager for each substituted class
can setup to keep object in memory
added PConcurrentHashMap
ClassInfo sun package class are marked storable
can give all accessed class using IFilePesistence#allClassInformation
new method IFilePesistence#isProxy
can record wrote object identifier when save called, useful for analyze of save operation
set of storage information by class name instead of class when class is not accessible

bugs corrections:

Consistency of ACID was not respected
save failure write nothing in file and keep it in its initial state ( state when opening data access session )
had useless write when read data record
now the class loader defined at set up is use everywhere
duplicate class definition error if multiple FilePersistence instance and same class to enhance
garbage some link was loose

21 November 2009

joafip-2.2.2b5 md5 sha1
joafip-java-agent-2.2.2b5 md5 sha1

Sources md5 sha1

https://joafip.svn.sourceforge.net/svnroot/joafip/branches/joafip-2.2.2b/tags/joafip-2.2.2b5

bug corrected:

crash safe recover does not work, backup file was corrupted
PArrayList contains returned wrong element index
PTreeList remove element did not work, addReplace was not implemented

enhancement:

add heap record to the "to be save list" done only once for the same heap record

2 November 2009

joafip-2.2.2b4 md5 sha1
joafip-java-agent-2.2.2b4 md5 sha1

Sources md5 sha1

https://joafip.svn.sourceforge.net/svnroot/joafip/branches/joafip-2.2.2b/tags/joafip-2.2.2b4

corrected bug:

export size of array was wrong
short data type read/write in file did not work
in already open/close exception for trace of previous closer/opener
$intercepted$ flag become true only is object loaded
TreeSupportSubIterator could not be enhanced
proxy byte code error for method with multiple long or double parameters

enhancement:

all added proxy methods have a name ending with $JOAFIP$
use a .lock file in storage directory
added unload proxy feature, not used yet
added FilePersistence#setClassLoader and FilePersistence#getClassLoader

19 October 2009

joafip-2.2.2b3 md5 sha1
joafip-java-agent-2.2.2b3 md5 sha1

Sources md5 sha1

https://joafip.svn.sourceforge.net/svnroot/joafip/branches/joafip-2.2.2b/tags/joafip-2.2.2b3

embedded asm library release 3.2
bytecode generation error corrected, discovered because 1.6 class verifier is more strict

xx/xx/2009

joafip-2.2.2b2 md5 sha1
joafip-java-agent-2.2.2b2 md5 sha1

Sources md5 sha1

https://joafip.svn.sourceforge.net/svnroot/joafip/branches/joafip-2.2.2b/tags/joafip-2.2.2b2

failure or null reference import corrected

27/06/2009

joafip-2.2.2b1 md5 sha1
joafip-java-agent-2.2.2b1 md5 sha1

Sources md5 sha1

https://joafip.svn.sourceforge.net/svnroot/joafip/branches/joafip-2.2.2b/tags/joafip-2.2.2b1

correction on listIterator#add for PLinkedHashMap and PLinkedHashSet

06/06/2009

joafip-2.2.2b0 md5 sha1
joafip-java-agent-2.2.2b0 md5 sha1

Sources md5 sha1

https://joafip.svn.sourceforge.net/svnroot/joafip/branches/joafip-2.2.2b/tags/joafip-2.2.2b0

enhancement had failure for method with long or double return type
bug : in crash safe mode if data flag exists and store data is loose restarted from an empty file. now check if data file exists when data flag exists, same thing for backup data flag file and backup data file.
stored enum state was not restored
code cleaning and refactoring
added export / import in xml format
byte code transformation option to cancel object persistence constraints
in proxy execution time optimization

29/03/2009

joafip-2.2.2 md5 sha1

Sources md5 sha1

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-2.2.2

garbage information was not saved at FilePersistence#close invocation
can now enhance package visibility method
in case of exception stop the background garbage thread

23/03/2009

joafip-2.2.1 md5 sha1

Sources md5 sha1

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-2.2.1

new object referenced by objets detached from root object now saved

23/03/2009

joafip-2.2.0 md5 sha1

Sources md5 sha1

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-2.2.0

Object detached from root not saved bug corrected. Discovered using PLinkedHashMap and PLinkedHashSet.
Can create multiple FilePersistence instance.
File persistence instance identifier added to prohibit storage of an object attached to a file persistence to be added in other file persistence.
Added close state to FilePersistence.
Object deep copy added to file persistence interface.
Trace last closer opener of file persistence.
A portion of asm 3.1 is embedded.

09/12/2008

joafip-2.1.0

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-2.1.0

joafip can now be setup using a properties file
stored enum state management correction
ClassNameRecord.toString null pointer exception in debug mode corrected
bug on sub tree set or sub tree map iterator corrected
Copier helper now force load of proxied object
count of object visited and modified bad management corrected

20/10/2008

joafip-2.0.0

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-2.0.0

speed and memory used optimizations
- do not keep temporary data in memory after object write or not management
- saver simplification
support standard serialization mechanism
do not longer manage ziped original value
class storage mode by annotation or programatively:
- annotations:
  - StoreSerializeInOneRecord
  - StoreSerializeAndZippedInOneRecord
  - StoreSerializeAndGZippedInOneRecord
  - StoreNoLazyLoad
  - StoreNotUseStandardSerialization
- programatively
  - setStoreSerializeInOneRecord
  - setStoreSerializeAndZippedInOneRecord
  - setStoreSerializeAndGZippedInOneRecord
  - setStoreNotUseStandardSerialization
  - setStoreNotUseStandardSerialization
setForceEnhance
- force class to be enhanced even if have final public method
by default substitution for java.util collection are not set, to activate needs call to setSubstitutionOfJavaUtilCollection
added exception
- FilePersistenceClassNotFoundException
- FilePersistenceDataCorruptedException
  - if CRC32 problems in heap file
  - if serialized data corruption
- FilePersistenceInvalidClassException
- FilePersistenceNotSerializableException:
  - do not follow joafip persistence rules
  - not serializable
check in case of class code changed making incompatible with previously stored data ( throw FilePersistenceInvalidClassException):
store object class signature
make control binary data store in file is fully used at reading ( to more, no less )
construct object with state in file without calling constructor ( proxied or not )
no more use cglib, proxy implementation using embedded asm 2.2 instead

08/10/2008

joafip-2.0.0rc3b1 ( release candidate 3 for 2.0.0 branch release 1)

https://joafip.svn.sourceforge.net/svnroot/joafip/branches/joafip-2.0.0rc3/tags/joafip-2.0.0rc3b1

bug correction on PlinkedHashSet, PHashSet, PLinkedHashMap, PHashMap

01/09/2008

joafip-2.0.0rc3 ( release candidate 3 for 2.0.0)

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-2.0.0rc2

write cache size no limited bug corrected: the maximum number of pages is for wrote and read pages
persistence rules mismatch for PLinkedHashSet corrected
refactoring for size and speed optimization
added PHashMap PHashSet
HashSupport optimization: used by PLinkedHashSet PHashSet PLinkedHashMap PHashMap
ObjectIoManager: optimized original value in array
refactoring for speed optimization
second cache on heap data manager removed
disable lazy load (proxy) for a class using annotations @StoreNoLazyLoad

26/08/2008

joafip-2.0.0rc2 ( release candidate for 2.0.0)

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-2.0.0rc2

file cache bug corrected
heap file bug on last record management corrected
added background garbage sweep, disabled by default.
new method for enable and disable background garbage sweep
file access performance increased using java.nio
( FileChannel instead of RandomAccessFile ) and no longer sometime write unchanged object
Forcing any updates to the files to be written to the storage device more effective
data file locked while it is used
field to persist test: the field by class contains now only field to persist
cache fields by object no longer contains all class' fields,only persistable fields. This avoids the persistable test for reading and writing object.
new conception for IHeapDataManager interface and implementation HeapMultiFileDataManager
HeapFileCheckerDataManager delegate to HeapFileDataManager and HeapMultiFileDataManager
checking same state
AbstractCreateSettedObjectIO#createReadingBody
have no longer data record identifier parameters.
implementation do not need associate object created with data record identifier.
storage by serialization ( zipped or not ) in one data record for class
having annotation @StoreSerializeInOneRecord, @StoreSerializeAndZippedInOneRecord ( with optional compression level),and @StoreSerializeAndGZippedInOneRecord
object with no generic save mode are not longer always saved.
this concern basic type, string, serialized.

10/08/2008

joafip-2.0.0rc1 ( release candidate for 2.0.0)

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-2.0.0rc1

test all collection serialization
StoreSaver simplified
enum management changed (major 2.0.0)
unique usage data identifier, identifier of garbaged data record is nether reused.
- done because:
    - restore from backup : id reconstruction for plenty of objects is very long
    - garbage : creation of data record without reusing old id needs
- data record identifier codded in 64 bits,
    64 bits -> 18.446.744.073.709.551.616
              18446744073709551616
    10 years -> 3650 days -> 87600 hours -> 315360000 seconds
    18446744073709551616 / 315360000 -> 58.494.241.735 update / seconds
    58.494.241 update per ms ! 64 bits is enough
bug on state change detection for object not using generic persistence corrected

10/08/2008

joafip-1.1.2b1.1

https://joafip.svn.sourceforge.net/svnroot/joafip/branches/joafip-1.1.2/tags/joafip-1.1.2b1.1

test all collection serialization
added check file open state or close state when calling IRandomAccessFile method
added assertion on heap record size at marshaling and uninstalling
ensure object with no generic save mode are always saved
had bug on random file cache ( page to write could be lost )
joafip collection iterator are Serializable
field that can not be null on collection annotated AssertNotNull
no more data record identifier number reuse, for each new record created a new number is used

18/07/2008

joafip-1.1.2

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-1.1.2/joafip

failed open with crash safe mode disabled and read cache enabled
correction of bug that make java heap space out of memory in case of crash safe mode disabled
failed delete not existing file, change to delete if exists
refactoring

08/06/2008

joafip-1.1.1

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-1.1.1/joafip

list iterator for PHashMapEntrySet
refactoring

27/01/2008

joafip-1.1.0

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-1.1.0/joafip

refactoring

11/11/2007

joafip-1.0.11

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-1.0.11/joafip

test added and correction for list and set ( // to sun implementation )
object created between getRoot/setRoot and save do not stay in memory ( prevent memory leak )
hash support memory size usage optimization
PLinkedHasSet have no more memory leak
visit for save: no more recursive method call to avoid stack overflow if stack too small

07/11/2007

joafip-1.0.10

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-1.0.10/joafip

storage of joafip storage management classe's is forbidden
no more methode close without action parameter for DataAccessSession
proxy load failed if out of access session ( out of the getRoot/setRoot save cycle )
update on java util collection replacement:
PtreeList support concurent iteration and modification and full List and Deque implementation
PLinkedList bug corrected and full List and Deque implementation
bug in Long storage corrected

02/11/2007

joafip-1.0.9

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-1.0.9/joafip

memory leak suppressed

25/10/2007

joafip-1.0.8

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-1.0.8/joafip

no more number of link information in data record
garbage management changed
can disable the crash-safe mode

16/10/2007

joafip-1.0.7

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-1.0.7/joafip

bug corrected on free record freeing
reference in data record no more contains class name identifier, class name identifier are on data record header with number of link

joafip-1.0.6

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-1.0.6/joafip

bug corrected on free record management

joafip-1.0.5

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-1.0.5/joafip

thread safe bug corrected
backup file access do not have read cache ( write only operation ) inprove performance
file opened in rws mode ( synchronize ) for data safe
fail open data access session if close pending
hash set do not accept null element
bug on enum storage corrected
garbage not stable, added garbage inactive mode
file direct : compute position in file to avoid useless call to seek

joafip-1.0.4

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-1.0.4/joafip

fileState from exception: change to static call, implemented in helper HeapFileState
append record to end of file, do not use file size, but byte position after last record
update file size according to byte position after last record
can setup cache at file persistence creation

joafip-1.0.3

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-1.0.3/joafip

file cache added
bug on garbage management fixed

third alpha release joafip-1.0.2

https://joafip.svn.sourceforge.net/svnroot/joafip/tags/joafip-1.0.2/joafip

garbage data record collector implemented
support persistence of enum type
management of object created by factory method and can return the same object for the same state [ example: Integer.valueof(...) ]