craftleft.gif (3002 bytes)QuickTable
Home | API | Tutorial | Download | Support | Contact topblack.gif (108 bytes)


QuickTable User Cp  |  Register  |  Members  |  Search  |  Help
    |- QuickTable Discussion Forums > General QuickTable Support/Help Post New Topic   Post A Reply
excessive cpu utilization printer friendly version
next newest post | next oldest post
Author Messages
SamB
Private

Gender: Male
Location:
Registered: Sep 2005
Status: Offline
Posts: 9

Click here to see the profile for SamB Send email to SamB Send private message to SamB Find more posts by SamB Edit or delete this message Reply w/Quote
Posted Tuesday, March 7, 2006 @ 15:24:13  

Ive done several test projects using quicktable with very happy results. Now
I am using quicktable as part of a java user iinterface for a data aquisition project I am working on. One of the requirements are display-only screens that need to be refreshed every 1.5 seconds. When Iran the first of whese screens, I noticed that my previously working acquisition program was starting to miss events, and it turned out I was seeing 100% cpu utilization, according to the standard windows xp task manager's graphical display. When I ran the single quicktable screen alone, being refreshed every 1.5 seconds, my cpu utilization
oscilated between 50% and 80%. I am using the standard database related refresh with no parameters. {note-
I am showing it in a jinternalframe}.

I looked further; firstly, I saw that a simple jtable with a moveable cursor-based data model (I copied out of a book) gave pretty much the same bad results. Then I coded up a much cruder jtable program using just a 2 dimensional object array, where the refresh consisted of java code to 1) issue the select statement 2) a for loop to load the resultset into an object array and 3) display jtable. This worked just fine;i.e. stayed under 10% always and under 2-4% most of the time. I therefore presumed that the culprit must be fancy jdbc cursor useage.

So I tried the same crude approach as described above with quicktable using an object array refresh and it works, almost. What happens is that for the first 15 to 20 seconds of display, quicktable's cpu utilization oscilates between 10 and 40%, then goes down to the under 5% level except for a blip every 20 or 30 seconds up to 20-30%.

I have several questions.......

1. Why, when using quicktable with constantly updated object arrays is there the high cpu utilization for the first
20 seconds;

2. what is causing the 20-30 second blip in the object array refresh as described above?

3. Is my observance that jdbc cursor useage, (at least for this circumstance), is a huge cpu resource hog
correct??
-if so is there any way, without source code, to force quicktable to not use cursors?? I assume it must
be able to run not using cursors in order to handle older drivers that dont support it.

I repeat my satisfaction to date with quicktables. Thanks for the very useful product.

Admin
Board Owner

Gender: Unspecified
Location:
Registered: Jul 2003
Status: Offline
Posts: 9

Click here to see the profile for Admin Send email to Admin Send private message to Admin Find more posts by Admin Edit or delete this message Reply w/Quote
Posted Tuesday, March 7, 2006 @ 23:59:50  

In order to better investigate this problem, I need the following information

1) version of quicktable you are using. Try the latest version to see whether there is any improvement.
2) jdbc driver used
3) database used
4)jdk used
5) using select query or stored procedure or resultset
6) approximate number of rows which you are trying to retreive
7)A small sample program which just uses quicktable on a JFrame and constantly refreshes reproducing this issue. You can email this program separately by email to support at quicktable.org

Once we get these information, we will further investigate this issue.

SamB
Private

Gender: Male
Location:
Registered: Sep 2005
Status: Offline
Posts: 9

Click here to see the profile for SamB Send email to SamB Send private message to SamB Find more posts by SamB Edit or delete this message Reply w/Quote
Posted Wednesday, March 8, 2006 @ 13:56:54  

With more testing, I am now totally confused and am no longer sure
this is a quicktable issue or not. I am now more suspicious of my spotty
knowledge of sql and jbdc more that anything else.

First, to answer your questions....

1. quicktable version 2.0.5.24, which unless youve updated in the last month or so is I believe the latest.

2. jsbc driver = microsoft sql server standard jdbc driver

3. microsoft sql server express, which Im trying out for the first time (so far I am begrudgingly pretty impressed)

4. jdk is 1.4......

5. using select query via setSelectSql();

6. I am selectng the top 500 rows of a 250000 row table (top =highest event sequence number, which is the primary key).

7. Ill send the code if this turns out to really be a quicktable issue, or if you want it anyway let me kow.

What Ive discovered so far....
1. using a jframe instead of internal frames doesnt do much; slightly better I think, but its not the issue.

2. I tried firebird/jaybird (latest stable version as of december) and got pretty much the same results; actually
slightly worse but I didnt spend much time tweaking, and may not have done the "limit 500" just right). I can
also try hsqldb and/or mysql if I have to.

3. Whats got me confused and self-doubting is that I made up a 500 record test table so that I could
eliminate the "top 500" phase out of my selectsql. When I did that, but still refreshing every 1.5 seconds, it worked; way under 10% useage always!!. I even made up a change simulator that would change records, adding and deleting a few every second so that the data was dynamic. Everything was still fine; way under 10%.
That says this might have something to do with the huge total size of my table; somebody somewhere (the db server, jdbc, quicktable, or a stupidity on my part) must be retreiving the entire table or something.

So, lets not go digging into quicktable quite yet. I need to check out a few things with my sql, my big table, expanding java memory space, etc first before I waste your time.

Thanks for the quick reply; Ill let you know what I see.

Admin
Board Owner

Gender: Unspecified
Location:
Registered: Jul 2003
Status: Offline
Posts: 9

Click here to see the profile for Admin Send email to Admin Send private message to Admin Find more posts by Admin Edit or delete this message Reply w/Quote
Posted Wednesday, March 8, 2006 @ 16:04:31  

This problem might be due to the way quicktable is loading the data.

In order for quicktable to load the visible data only (without loading the whole 250000 records during refresh), it needs to know the total number of rows in result. so first it automatically creates a row count sql as "SELECT count(*) FROM xyz" and executes this and gets the count. May be the automatically created rowcount sql is bad or inefficient. If you enable debug using dBTable1.debug=true, then quicktable will print all information about the queries executed and any failures. This is the first thing you have to try.

I feel that in your case, the automatically created rowcount sql is failing. So quicktable fetches all 500 records for every refresh.

Since you just need 500 records, set the rowcount sql such that the sql returns one record with the value 500.
dBTable1.setRowCountSql("SELECT 500"); //sybase
dBTable1.setRowCountSql("SELECT 500 from dual"); //oracle
I do not know how the sql will look like for SQLserver.

This should fix your problem.

SamB
Private

Gender: Male
Location:
Registered: Sep 2005
Status: Offline
Posts: 9

Click here to see the profile for SamB Send email to SamB Send private message to SamB Find more posts by SamB Edit or delete this message Reply w/Quote
Posted Wednesday, March 15, 2006 @ 20:44:56  

First the good news; yes, you got it exactly right. The main problem was the rowcount sql. When I tried
"select abs(500)" in sqlserver, a very large part of the problem went away!!!! thanks.

However I still see a 20-30% cpu useage for this quicktable screen, and after further experimentation,
including trying java 1.5, I suspect it has to do with the fact I am using JInternalFrames. What I see is..

a. If I change the screen to a jframe, then no problem; normal cpu useage is well under 5-10%.

b. with Quicktable using JInternalFrames I dont have to be doing continuous refreshes to get this problem; just having the quicktable screen visible and refreshes stopped seems to cost me 20% of my cpu.

c. I made up a simple jinternalframe quicktable array screen that just displays a 10x10 unchanging array of "fubar" strings and that alone will continuously use up 15% of my cpu until I hide the screen.

d. None of my non-quicktable screens, (including a couple of jtable screens I haven't converted to quicktable yet) have this problem (all under 5%).

So my question is; is there anything unusual happening in quicktable when it is running in a JInternalFrame??
A couple of previous threads alluded to forced garbage collection, but from what I read only on closing; My issue is more continuous; I can make the 20% cpu jump come and go just by hiding/unhiding the quicktable screen with another screen.

Thanks again for the help; your support is great.

SamB
Private

Gender: Male
Location:
Registered: Sep 2005
Status: Offline
Posts: 9

Click here to see the profile for SamB Send email to SamB Send private message to SamB Find more posts by SamB Edit or delete this message Reply w/Quote
Posted Wednesday, March 15, 2006 @ 20:58:18  

First the good news; yes, you got it exactly right. The main problem was the rowcount sql. When I tried
"select abs(500)" in sqlserver, a very large part of the problem went away!!!! thanks.

However I still see a 20-30% cpu useage for this quicktable screen, and after further experimentation,
including trying java 1.5, I suspect it has to do with the fact I am using JInternalFrames. What I see is..

a. If I change the screen to a jframe, then no problem; normal cpu useage is well under 5-10%.

b. with Quicktable using JInternalFrames I dont have to be doing continuous refreshes to get this problem; just having the quicktable screen visible and refreshes stopped seems to cost me 20% of my cpu.

c. I made up a simple jinternalframe quicktable array screen that just displays a 10x10 unchanging array of "fubar" strings and that alone will continuously use up 15% of my cpu until I hide the screen.

d. None of my non-quicktable screens, (including a couple of jtable screens I haven't converted to quicktable yet) have this problem (all under 5%).

So my question is; is there anything unusual happening in quicktable when it is running in a JInternalFrame??
A couple of previous threads alluded to forced garbage collection, but from what I read only on closing; My issue is more continuous; I can make the 20% cpu jump come and go just by hiding/unhiding the quicktable screen with another screen.

Thanks again for the help; your support is great.

Admin
Board Owner

Gender: Unspecified
Location:
Registered: Jul 2003
Status: Offline
Posts: 9

Click here to see the profile for Admin Send email to Admin Send private message to Admin Find more posts by Admin Edit or delete this message Reply w/Quote
Posted Wednesday, March 15, 2006 @ 22:04:35  

Please reproduce the problem in a small sample program with JInternalframe and send it to us, we will look into this.
SamB
Private

Gender: Male
Location:
Registered: Sep 2005
Status: Offline
Posts: 9

Click here to see the profile for SamB Send email to SamB Send private message to SamB Find more posts by SamB Edit or delete this message Reply w/Quote
Posted Monday, March 20, 2006 @ 16:45:38  

What I have found so far by disabling features in my user interface one by one is that quicktable is for some reason in conflict with a swing timer used to update my Jdesktopmanager's background picture (to show device status changes, etc). If I turn the timer off, the problem goes away; if I turn it back on, I lose my 20% of cpu but only when displaying a quicktable screen. Since the desktop manager code isnt mine, I have to wait for the person who did write it to get back to me in order to check any deeper into this. This might be a while. Will get back to you when I can do that.
Post New Topic   Post A Reply Jump to:
Contact Us | QuickTable - A Java DBGrid | Privacy Policy All times are GMT -5 Hours.
Welcome to QuickTable Forums, Guest!  
Login
Username :
Password :
In order to fully utilize the abilities of this board, you are required to register as a member. Registration is free, and allows you to do lots of things including turning on or off certain features of this board. Register now!
Powered by CuteCast v2.0 BETA 2
Copyright © 2001-2003 ArtsCore Studios