QuickTable | Thread - Loading huge tables cause Out of memory error

QuickTable
Home \| API \| Tutorial \| Download \| Support \| Contact

QuickTable

User Cp | Register | Members | Search | Help

\|- QuickTable Discussion Forums > General QuickTable Support/Help	Post New Topic Post A Reply
Loading huge tables cause Out of memory error	printer friendly version next newest post \| next oldest post

Author	Messages
Mathieu Private Gender: Male Location: Leiden (Holland) Registered: Aug 2011 Status: Offline Posts: 1	Posted Friday, August 26, 2011 @ 07:28:53 Hello, I'm working with QuickTable 2.0.5 and both MySQL and Postgres databases (recent versions). It is working quite well, even with databases with 200.000 records But I experience troubles with bigger databases. For example, I tried to access a MySQL database consisting of 16 million records. Accessing this database in MySQL workbench is no problem at all. But QuickTable can't load it. All I get is (with dBTable1.debug=true): Executing row count sql=SELECT COUNT(*) FROM words Time taken to execute rowcount sql=9688ms Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space at com.mysql.jdbc.Buffer.getBytes(Buffer.java:128) at com.mysql.jdbc.Buffer.readLenByteArray(Buffer.java:286) at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1019) at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:305) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1326) at com.mysql.jdbc.MysqlIO.sqlQuery(MysqlIO.java:1225) at com.mysql.jdbc.Connection.execSQL(Connection.java:2278) at com.mysql.jdbc.Connection.execSQL(Connection.java:2225) at com.mysql.jdbc.Statement.executeQuery(Statement.java:1163) at quick.dbtable.t.for(DBTableModel.java:964) at quick.dbtable.t.a(DBTableModel.java:610) at quick.dbtable.DBTable.a(DBTable.java:2685) at quick.dbtable.DBTable.refresh(DBTable.java:2847) at screens.AnalyzedWordformsFrame.open(AnalyzedWordformsFrame.java:68) at screens.StartScreen.actionPerformed(StartScreen.java:61) at javax.swing.JComboBox.fireActionEvent(Unknown Source) at javax.swing.JComboBox.setSelectedItem(Unknown Source) at javax.swing.JComboBox.setSelectedIndex(Unknown Source) at javax.swing.plaf.basic.BasicComboPopup$Handler.mouseReleased(Unknown Source) at java.awt.AWTEventMulticaster.mouseReleased(Unknown Source) at java.awt.Component.processMouseEvent(Unknown Source) at javax.swing.JComponent.processMouseEvent(Unknown Source) at javax.swing.plaf.basic.BasicComboPopup$1.processMouseEvent(Unknown Source) at java.awt.Component.processEvent(Unknown Source) at java.awt.Container.processEvent(Unknown Source) at java.awt.Component.dispatchEventImpl(Unknown Source) at java.awt.Container.dispatchEventImpl(Unknown Source) at java.awt.Component.dispatchEvent(Unknown Source) at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source) at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source) at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source) at java.awt.Container.dispatchEventImpl(Unknown Source) I also used setRowCountSql(...), but it doesn't help. I understood from other posts that data is only loaded when it's needed (when the user scrolls down and such), so how come I run out of memory now? Do you guys have experiences with huge databases and QuickTable and do you know any solution?
Admin Board Owner Gender: Unspecified Location: Registered: Jul 2003 Status: Offline Posts: 9	Posted Sunday, September 4, 2011 @ 21:54:02 quicktable executes the query once and as users start scrolling down, it loads the data from resultset. If you scroll all the way down or if you are using load all, then all records will be loaded and you will get out of memory. Quicktable just delays the data loading, but not entirely takes away the problem. lets say you show 16 million records in a grid, it will be difficult for users to find the record which they are looking for by scrolling down the rows and checking one by one. usually users will enter a search criteria, so it will be a managable number. In addition you can also implement paging, instead of showing all records. show first 1000, then they will click next page and then you can load the next 1000.

Post New Topic Post A Reply

Jump to:

All times are GMT -5 Hours.

Welcome to QuickTable Forums, Guest!

Username	:
Password	:

In order to fully utilize the abilities of this board, you are required to register as a member. Registration is free, and allows you to do lots of things including turning on or off certain features of this board. Register now!