First things that come to my mind are referencial integrity, schema evolution, etc.
The VM's strong typing does referential integrity.
DBMSs don't help you much with schema evolution. You have to write the migration scripts anyway. --
KlausWuestefeld
!!!!
Won't a sufficiently complicated database application written using
Prevayler contain an ad-hoc, informally-specified bug-ridden slow implementation of half of a
DBMS?
No. Your database didn't allow it so you probably didn't consider this an option but, now, with
Prevayler, you are finally free to use object libraries! @;) --
KlausWuestefeld.
Are the best practices one finds in today database theory all best forgotten?
All the ones spawned from the problem of paging memory blocks to disk, yes. --
KlausWuestefeld
I think one of the reasons that
Prevayler can be so radically simple is that it doesn't deal with the issue of making data remotely accessible. A database server, apart from providing a reliable storage mechanism for data, necessarily has to expose this to remote clients.
Prevayler's striking simplicity is possible because it assumes that the data will be consumed only by the one, local application. It's likely that at some point during the lifetime of a
Prevayler-based application, there will be another application that needs to use its data. --
MichaelPrescott
Yes. With a database you hava a DATA server. With
Prevayler, you have an OBJECT server. The application is free, therefore, to decide how it will let its clients and other systems access those objects. Some options include RMI, CORBA, XML, Sockets, JSP, Servlets, etc.
For web applications, you have an extremely simple and fast architecture which is JSP/Servlets accessing the objects directly in the same VM. --
KlausWuestefeld
How is record locking implemented?
You now have objects, not only records. See:
WhatAreTheCodingRestrictionsMyClientClassesHaveToObey?
Has anyone tried IBM's open-source ICU4J unicode compressor? For string-heavy applications using Latin charsets, it looks like it could save as much as 50% of the space. It claims it works best for small to medium sized strings.
Here's the API: http://www-124.ibm.com/icu4j/doc/com/ibm/icu/text/UnicodeCompressor.html
JonathanCarlson
Any plans of integration with EJB containers for persistence?
That might be good marketing. Technically, though, it would be a step backwards. --
KlausWuestefeld and
CarlosVillela
Your idea is really the coolest one of 2002 so far!
Thanks, we're glad you liked it! @:) --
PrevaylerTeam
HowCanIMeasureTheAmountOfMemoryNeeded?
Wouldn't it make sense to use some persistent collection (BTree?) that uses the filesystem to temporarily swap out objects? This would reduce the memory requirements...
That would be as complex to implement and as slow as an object database. --
KlausWuestefeld
Prevayler for the
PalmPilot.
I'm realy happy you did this @:) I just was a month away from starting exactly this project as I'm about to build a massive multi user world for which I considered this persistence schema the best suiting.
Best Regards, angel'o'sphere aka angelo.schneider@oomentor.de
*sigh* I've been looking at the demo's, tests and implemntation classes all day long, and I still can't figure it out. It's giving me a headache. Would it be possible to write a simple document on how to design an application to store its data using
Prevayler? I really can't figure out what class to use how and where, and how to handle everything :(
Take a look at
GettingStarted, especially the articles. There's a nice intro by
CarlosVillela --
KlausWuestefeld
The comparison of prevayler with SQL-Databases is misleasing. The latter supports
concurrent writers while prevayler does not. Currently I cannot see how
Prevayler
addresses concurrent updates from multiple processes (VMs).
As of release 1.03,
Prevayler does provide concurrent transaction logging. Multiple processes (VMs) will access the
Prevayler VM normally, as if they were clients. --
KlausWuestefeld
OK. Assume this "classical" problem: Two users of an ERP system try both to
update the address-record of a customer. Both are connected by separate
processes to the db-storage. With an SQL-
RDBMS *one* solution is to open
transactions for both clients. Ony a single one will succeeed.
So how does
Prevayler support shared access in detail. Are there any
design documents/concepts?
No. You actually have to know your OO and your Java. --
KlausWuestefeld.
See:
WhatAreTheCodingRestrictionsMyClientClassesHaveToObey
How can one make the
Prevayler compatible with languages like Delphi or Kylix ?
Do they have some sort of object serialization? If so, then an
ObjectPrevalence layer can be written for them.
Schema evolution is the biggest problem. I am trying to port one of my applications to use
Prevayler and it is difficult to work in an iterative manner as all data has to be reloaded again and again using crude tooling. I decided to start building the business classes and commands based on my spec and then adding the servlet/velocity layer. Using a DB is not cool (but
SimpleORM helps there (www.simpleorm.org)) but allows for schema modification and good reporting.
Well, I will do both versions and see what is the most appropriate for the small product of mine.
If I get it done, I'll post it somewhere in the next weeks.
--
PhilippeBack
Cool. I suggest you start using automated testing. That way, you only have to reload your data at release time and not during development. --
KlausWuestefeld
Next comment is that the system becomes a very big FacadeClass with too many methods relaying the work to CommandObjects.
I would like to be able to partition those methods in several groups:
- to allow separate developers to work on their part
- to avoid a big class that does everything on the BusinessObjectModel
I takes a while to get used to it (half a day or so) but it can prove useful (for prototyping, it's sure that I can win some time). For production development, that's another story... let's give it a chance though as the idea is clever (kind of like a SmallTalk image for BusinessModelObjects).
--
PhilippeBack
"I would like to be able to partition those methods in several groups"
Just make BigFacade return ^SmallFacade1, ^SmallFacade2, etc, with commands/transactions grouped according to their functionality. You can even have business objects returning the Commands/Transactions that operate on them. --
KlausWuestefeld.
What happens if I run out of RAM unexpectedly? Is graceful degradation just swapping out to Virtual Memory until the RAM delivery van comes around again?
Yes. Your system will become pityfully slow, though. See the
PrevalentHypothesis. --
KlausWuestefeld
DBMS's use raw partitions to gain better control over physically writing to disk -- can
Prevayler use a DB to store the log?
-- KenHorn
The current implementation cannot usa a DB to store the log. I think the JDBC overhead is greater than the Java file-writing overhead anyway, though. --
KlausWuestefeld
Also, disk is significantly cheaper than RAM, and is getting cheaper much faster. If I've got a 20GB database, can I really afford 25GB in RAM to use
Prevayler? I think my SA's would just laugh.
This looks useful for small to medium data requirement apps, but I can't see it scaling without transparent paging to disk (which OS may provide enough of).
-- KenHorn
How much does 25GB of RAM cost? US$ 3000? Can you afford that?
I can see a lot of advantages in using this for small projects. However I am still not sure how to use it in a web application architecture. Since a servlet based architecture (jsp, velocity, etc) is multi-threaded, what are some best practices about building access to the Prevalent objects? The simple approach is to use an exclusive lock around the request processing. Since data access is so fast it might be possible. To increase concurrency one would need more complicated locking around each usage of a "prevaylerbase" controlled object, but this makes things harder to set up. Any comments/experiences would be interesting to hear. - Andrew
See:
InstantaneousTransactions,
WhatAreTheCodingRestrictionsMyClientClassesHaveToObey?
How is
Prevayler related to and what are the advantages of
Prevayler over the RAM I/O Project (http://www.eecs.umich.edu/Rio/) (1996) which suggests a similar solution?
How can you keep bugs from contaminating the Data in a system using
Prevayler? If the whole system crashes, my data would be safe in a conventional storage system. What happens if my objects are RAM-persistent such as in
Prevayler? My application may be running for a while and messing things up before crashing. There's no place where my data would be safe. A bug in my software may cause the corruption of data which, in a conventional persistent system, could be more reliably protected.
That is a myth. "How can you keep [application] bugs from contaminating the Data in a system" using a database? --
KlausWuestefeld
Do you guys have any reference implementation?
Prevayler 1.00.xxx is the reference implementation. --
KlausWuestefeld.
What is the main difference with the existing HSQL database, which is tiny SQL compatrible database written in pure Java with JDBC interface. It also can keep tables, indexes, etc. in memory, make snapshots and transaction logging...? Another clone?
Prevayler is MUCH simpler (therefore much more robust), orders of magnitude faster (no JDBC overhead), and works for objects, not only for records. --
KlausWuestefeld.
See:
PrevaylerIsNotADatabase.
Doesn't the system have to stop and restart to apply code changes even if the data and its structure is unchanged?
With
Prevayler, new commands (soon transactions or queries), new business logic code, or new client code must be compiled into a new executable. To substitute executables, one must stop the old executable--preferably after a snapshot--and start the new executable, which must load data from a snapshot and log files.
Compare this to SQL, where the database server (
Oracle, Sybase, Postgresql, ...) is a stable, long-running process independent of application code changes. With SQL, I can change reporting requirements, transactional data sources, etc. without shutting down the database server. For example, my web shopping cart can continue to accept orders as I change my order fulfillment system. --
JoelShprentz
We can just load the new version of the code on a replica and route the clients transparently to it. See:
DoesntTheSystemHaveToStopInOrderToProduceAConsistentSnapshot?
Also, Java is more naturally run on a virtual machine rather than compiled to an executable. If we don't want to use a replica, therefore, we can use Java's dynamic class loading in association with lazy migration to do a live upgrade of the system. That way, our web shopping carts can continue to accept orders as we change our SHOPPING CART system! @8) --
KlausWuestefeld.
'That is a myth. "How can you keep [application] bugs from contaminating the Data in a system" using a database?'
This is not a myth. You can actually use stored procedures and constrains to protect the data in a
DBMS. Also I can think of situation on which roll-back is really usefull. Ries
Did your stored procedures and constraints come as part of your
DBMS or are they part of "the application"? @;) Also, I didn't say rollback is useless, I said
RollbackIsNeedless. --
KlausWuestefeld
I discussed this with coworkers and some just cannot get the point.
Let me quote a line from 'The Matrix': "There is no database"
"You have to understand, most of these people are not ready to be unplugged. And many of them are so inert, so hopelessly dependent on the system that they will fight to protect it." --Morpheus in "The Matrix"
We ran across
Prevayler last week and were surprised to find a lot
in common with a system we are developing. Chief differences:
+ Commands (we call them updates) are implicit; we trap mutative
operations and use the Java reflection API to extract the methods
and their arguments for later reapplication. (We rely on
AspectJ
to help with this magic.)
+ We use XML to serialize both the operation lists and the state
snapshots, not Java serialization.
+ We can group multiple mutative operations into a single update.
In the context of this update, local state appears to have
changed -- accessors return the changed values -- but uncaught
exceptions cause the state to be rolled back to the point before
the update was initiated. Normal return from the update context
commits the changes atomically. (Again we rely on
AspectJ to
make this work.) As a by-product of this, we can offer robust
disconnected operation.
But for the most part the approaches are strikingly similar. No
database, all state must be in memory, changes are logged with
a repository before committing to in-memory state.
--tim@peierls.net
To answer the RAM cost question. For 1GB of Sun RAM, for an x000 range, is $4,000 -- $100,000 for RAM is pretty expensive, plus in order to use this much a BIG box is required, at least an Enterprise 6000. When this maxes out because of extra data, upgrading a whole box instead of adding a disk to the array/SAN.
-- KenHorn
I think there are alot of really excellent things about
Prevayler... It's fast, easy to learn and use, small footprint, and so simple that it doesn't come with (or need) an ant build. These are great things. But, alot of what I'm reading here about RAM vs disk costs and the associated limitations (such as the admitidly "pitifully poor performance" of
Prevayler when the OS runs out of RAM) lead me to think that a hybrid solution is probably the most logical for larger projects.
Fer Instance. Lets say I have an online shop that sells books. To try to cache the entire book catalog in the same memory space as the app server and application is just eating up resources that most of the time aren't used (most people will likely be interested in a set of books way more than another set). This (as well as full text searching, etc) are where I think you're still going to need a
RDBMS. But why does it mean that you can't store the "top 1000" selections in ram, as well as say, the customer information (so they can quickly login). This is all hypothetical, but I guess what I'm trying to say is that I can see alot of instances where you would want to load and unload a subset of data into something like
Prevayler. Why does it have to be one or the other. Isn't there room for the 2 to peacefully coexist? I'm convinced it's fast, but not convinced that for a number of applications it wouldn't grow unanageably large and eventually cause OutOfMemoryErrors or suffer from very slow performance. How about storing a rolling set of commands and archiving the previous ones to the database? Does the
Prevayler team see any use for any types of hybrid approaches? Has the
Prevayler team done any testing to determine and recommend strategies for developing very large catalogs or other similar applications? Simply saying "your modeling sucks" really doesn't solve this problem
--Joe Maisel
I think all of this worrying about infinitely large datasets being stored in memory is a bit on the ridiculous side... Of course you wouldn't want to store hundreds of gigabytes (or more) in memory on a single machine. That wouldn't be cost effective, and since in this game it is all about cost vs. benefit it would be almost ludicrous to spend the kind of money to store a very large system in main memory.
That being said. There are many options for large data sets (as Joe said). One is using a
RDBMS or ODBMS. Another is "roll your own" solution maintaining your "business" data in memory and your "non-business" data in a slower to access medium (the file-system, tape, shelf 552 box 27, etc). Prevalayer serves a purpose in that it allows projects that need a simple way to have persistence from having to worry about having an external database and communicating with it.
--Brian S. Lloyd-Newberry
Why was this site unavailable for almost 24 hours after it was posted on slashdot? That doesn't sound like a highly responsive data store.
How foolish. This guy incorrectly assumes the
Prevayler site runs on
Prevayler and draws his conclusions from that. --
KlausWuestefeld
Where's my relational algebra? (e.g. http://www.cis.ohio-state.edu/~gurari/course/cis670/cis670Ch4.html)
Prevayler does not match the capabilities offered by a relational database. I'd say you're comparing apples to oranges.
Yes, it is comparing apples to oranges. You keep using your
RDBMS. My clients average database size is 1 - 50mb.
Prevayler is absolutely perfect for that.
They also don't mind the fact that development time is cut in half. - Sean
A highly responsive data store also needs a highly responsive pipe to pump data through. The slashdot flooding effect doesn't always mean that a server is unable to handle the load, but the internet connection too.
This site does not run on
Prevayler but yes,
Prevayler actually raises the performance bar on all other components in your architecture. --
KlausWuestefeld
I'm really loving prevayler. Although I must make one comment... I wouldn't use it for a project where the DB size was going to be over 4gb... The comment made is "How much does ram cost? Can you afford <insert amount here> for 32gb of ram".. The rebuttle to that is "Yes I can afford the ram. No I cannot afford the server to stick it in." A server which can hold 32gb of ram isn't an intel based machine, not one you buy at your local computer market anyway :)
Until the 64bit machines become common place, and desktops/servers in our sme's are > 4gb standard, I will only use prevayler for smaller apps (Hell. some of my "big" apps, only use 50mb of ram for the DB anyway, so its not really much of an issue)
- David
How does
Prevayler deal w/ an evolving object model? For instance, say I have a Person class w/ two fields: 'firstName' and 'lastName'. I persist an object of this class using
Prevayler. Down the road I update Person to have a 3rd field: 'middleName'. What will happen when I try to pull the older instance out of
Prevayler (the one w/o 'middleName') and cast it to my updated version of Person (the one that now has 'middleName')?
My understanding is that
Prevayler uses simple object serialization for taking snapshots. When I tested this (w/o
Prevayler...just serialized to text file) I got a "InvalidClassException: Person, local class incompatible: stream classdesc serialVersionUID = -5507413414685982915, local class serialVersionUID = -8332091257996572949"
See:
SchemaEvolution.
One issue that necessarily happens in a prevalence system is that the operations have to be completely serialized, which creates a potential bottleneck at the prevalence layer that a persistence system doesn't have.
In
Prevayler, only one command can be executing in the entire system at a given time. Many
DBMSs can update distinct rows concurrently - and pretty much all of them can at least update different tables concurrently.
Prevayler requires each command to be completely written to the transaction log and then completely executed before the next one can start. This could be pipelined a little (once the first is written and begins executing, the write for the next could start), but that's as far as it can go in an automated environment.
On the whole, it probably doesn't matter because of the improved performance you get from getting rid of the object/relational mapping code, but it's something to consider.
Running two systems in parallel, with the "hot" system *not* writing the transaction log could maybe improve this, too.
Just some random thoughts --- Scott Brickner
In
Prevayler 1.03, transactions are logged in parallel on several files. You forget that databases have EXACTLY the same thing: the redo log. Databases still have to write to the rollback segment and to the actual data-blocks (DBWR process in
Oracle, for example). They can do that in parallel, as you mentioned, but
Prevayler doesn't even have to do any of that. The
Prevayler equivalent is the Snapshot, which can be taken at an off-peak moment or even on a replica. See
InstantaneousTransactions. --
KlausWuestefeld
Very enthusiastic about the concept of
Prevalence. Tested Prevalyer last night and coded a working demonstration up in 30mins. One question..
If you are coding nearly any real world system your business objects are likely to change fairly frequently. I coded a small business object and populated my system, then brought down the system and added a toString() method to the object. When I tried to re-populate the system Serialization complained, I presume because the object it was trying to de-serialize to was not the same as the object it had serialized from (my business object had changed). Any advice on avoiding this?