Skip to main content

Mustang feature: Table Sorting

38 replies [Last post]
Anonymous

Table Sorting

We would again like the communities feedback on another long overdue
mustang feature we are working on. It's the ability to sort and filter
a JTable. At a later date we'll present the working API, but for now
we want feedback on how developers commonly use JTable.

We have two options with how the work is done.

One option is similar to the TableSorter code shipped with the JDK and
in the tutorial. That is, we wrap your TableModel in another model
that adds the ability to sort. One ramification of this approach is
that all rows that JTable maintains would no longer be in the
coordinates of your TableModel. For example, if you had code like:

<br />
int[] selectedRows = table.getSelectedRows();<br />
for (int i = 0; i < selectedRows.length; i++) {<br />
   myTableModel.doSomethingWith(selectedRows[i]);<br />
}<br />

you would not get the expected results, instead you would have to do:

<br />
int[] selectedRows = table.getSelectedRows();<br />
for (int i = 0; i < selectedRows.length; i++) {<br />
   myTableModel.doSomethingWith(table.viewToModel(selectedRows[i]));<br />
}<br />

Similarly if you knew you wanted to select a certain row that was in
terms of your model you could no longer do:

<br />
table.setRowSelectionInterval(row, row);<br />

instead you would have to do:

<br />
table.setRowSelectionInterval(table.modelToView(row),<br />
                               table.modelToView(row));<br />

If we go this approach we'll be sure and persist the selection and
variable row heights, in terms of the model, when the sort order
changes. This would mean any time you sort any ListSelectionListeners
will be notified of a selection change, but in terms of the model the
selection will not have changed.

The second approach is to change many of JTable's row based methods to
remain in terms of the supplied model. This has quirks too, in
particular if there are places where you really want something in
terms of the view you would have to convert. For example, if you
wanted to select the first visual index you may have code like:

<br />
table.setRowSelectionInterval(0, 0);<br />

that would need to be converted to:

<br />
table.setRowSelectionInterval(table.viewToModel(0),<br />
                               table.viewToModel(0));<br />

Possibly the biggest quirk with this approach is the range as reported
by the ListSelectionEvent. If we go this route the selection model
will be in terms of the model, but this means as the user is dragging
a selection the range of the selection change may be very large. For
example, lets say the first two visual rows of the table map to model
indices 100 and 5000. If the user drag selected the first two visual
rows the ListSelectionListener would report a first index of the
changed region as 100 and the last index as 5000.

Additionally selection modes become a bit tricky. Should the
selection mode be contiguous in terms of the view, or in terms of the
model?

As you can tell, each approach has it's own set of problems.
Comments? What would you like to see?

Thanks!

-Scott

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Scott Violet

Moss,

> I'm a little surprised that you seem to be focused on what convention to
> adopt for row indexes. Hopefully that's just because all the bigger
> issues are already solved and you're just not telling us!

The rationale for that is that if done correctly very few developers
should have to care about the actual implementation or format that
sorting takes. Ideally it's just matter of make this JTable sort.
Where developers will have to deal with sorting is in how the selection
indices change. Don't get me wrong, I know there will be developers
that want to customize the sorting algorithm, provide custom
Comparators, you name, but those are the minority.

> PS. what I'd _really_ like to see is a more ambitious effort to come up
> with a more usable TableModel/JTable/JTableHeader/TableColumn
> interaction. The current scheme is a good simple foundation with an
> overly constraining model/view interaction. What do people think; can we
> wait for Sun, or should there be new efforts to do something better?

I'm certainly interested in your thoughts in this area, but that sounds
like it would be better served in another thread.

-Scott

kleopatra
Offline
Joined: 2003-06-11
Points: 0

Hi Scott,

>
> If we go the route of having coordinates remain in
> terms of the view
> developers will have to change their code.

my first impulse was to write: "only the unfortunates who forgot to read the API doc" - as I remember the doc was _very_ clear that all indices in all JTable related methods are always in view coordinates ... But surprise, surprise, I couldn't find it in the current api. Had to dig back until 1.3:

" The general rule for the JTable API and the APIs of all its associated classes, including the column model and both the row and column selection models, is: methods using integer indices for rows and columns always use the coordinate system of the view. There are three exceptions to this rule: ..."

Now - why was it removed from the doc? How are row coordinates internally handled? Is there any assumption that rows are in model coordinates?

After checking some of my own classes - I always assumed that the 1.3 convention to be active. So unsurprisingly, my preference is: stick to the above rule - make everything view-coordinate (contrary to what I said last Wednesday, I got a bit confused sorry ;-)!

Jeanette

Scott Violet

Hi Jeanette,

I've no idea why those lines were removed. I could try and dig back
through the SCCS history, but that would be painful.

Thanks for the feedback!

-Scott

scott.violet
Offline
Joined: 2006-02-17
Points: 0

Has been integrated and is now availble on the jdk dev site (http://jdk.dev.java.net). A thread has started on the mustang forums relating to sorting/filtering: http://forums.java.net/jive/thread.jspa?messageID=16949&#16949 . There is a link to the API there. If you have the time to look over the API we would love your feedback!

It would be good if we could keep the comments in one place.

-Scott

carloscs
Offline
Joined: 2003-08-11
Points: 0

Seems like now I have a real reason to finally try mustang :)

I'll have to find the time somewhere...

seane
Offline
Joined: 2005-06-27
Points: 0

Hi all,

I know it's a little late to respond, but I try.

In my opinion sorting should be done on the model and then view should reflect the changes in the model, this way the consistency of the model/view is at the maximum, and there wouldn't be no conversion from model to view or from view to model needed. This will reduce the chances of any error in the application development process as well.

I know some might say that could be expensive in terms of memory and cpu, but the result will justify the expenses. I have developed a few application using this approach and even with tables of over 1000 rows/6 columns and fairly complex comparison (of dates as a string and not a Date object), the sorting time was acceptable.

The approach I took was, I attached a sorter to each table column header and provided a comparator for each column to the column header listener and the rest was just the implentation of the compartor and compartTo, which could be left to the developer.

Sean

Scott Violet

Sean,

Thanks for the feedback, and sorry for the delay in responding. JavaOne
really impacted my ability to do anything else this year:(

If I'm to understand your suggestion you are advocating doing sorting at
the model level. Developers can do this with the API that has been
added in mustang. This is a bit unsavory though in that it requires
each developer with a custom TableModel to do the sorting themselves,
which is why the sorting API also allows for sorting to be completely
handled for you.

Thanks for the feedback.

-Scott

Scott Violet

The resounding response to this appears to be people want JTable to keep
row coordinates in terms of the view. I must admit I'm perplexed by
this response. It implies you guys want to do conversions like:

[code]
int[] selectedRows = table.getSelectedRows();
for (int i = 0; i < selectedRows.length; i++) {

myTableModel.doSomethingWith(table.convertRowIndexToModel(selectedRows[i]));

}
[/code]

all over your code. Bleck;)

Anyway.

I wanted to provide an update as to how it's all coming.

Many people also communicated they like the decorator approach, similar
to the TableSorter we ship with the JDK. In evaluating the feasibility
of this a couple of things are leading us from this exact approach. In
particular the expectation with variable row heights and the selection
model is that it persists in terms of the model between sorts. For
example, lets say that visually row 0 is selected and that row 0
corresponds to a location in the model of 5. If you sort the table such
that model row 5 is at the visual position 10 you would expect row 10 to
be selected. You can see similar behavior in Windows Explorer when you
sort. Variable row heights are the same. An implication of this is
that JTable must some be cognizant of the sorting and so the wrapping
approach falls over.

Similar problems exist in correctly generating model mutation events
with the decorating approach. For these reasons we are heading to an
approach similar to that of JDNC.

-Scott

zet
Offline
Joined: 2003-06-11
Points: 0

Hi,

some feedback from someone that knows two sorting tables, see http://cuf.sourceforge.net for the code.

A)
The inherent problem between sorted/filtered (view model) and original (backing model) can't be solved without a compromise.

Perhaps the easiest compromise is to enhance
JTable.getSelectionModel()
et.al. with a parameter "TableModel matchingTableModel" and hand back a selection model based on that model.
This would also require some API addition in the table model to support stacking of table models.

I'm not sure what the default (what you get by the current call) should be, but i assume that the backing model index list selection is better for backwards compatiblity.

B)
The default (pluggable) table sorter should implement a stable sort, like the original TableMap implementation.

C)
Sorting multiple columns should be supported, perhaps with CTRL-click meaning "add this column to the sorting".

D)
Even if this is not sorting/filtering related: please add a "fit column with to max column with" functionality, including the double-click on the space between the header rows to trigger it.

E)
The selection must be stable after a sort order change, and be still visible.

F)
Don't change too much :-)
A litmus test could be if a ValueModel data binding doesn't need changes to get the benefits of a (simple, single selection) sorting.

G)
Some (plugable) table header renderes are needed, and please match the real Windows look in the Windows L&F ;-)

Bye,

Jürgen

dstrupl
Offline
Joined: 2003-06-11
Points: 0

Hello,

I vote for keeping the API similar to what you have for columns. Means do the sorting on the view level and provide the convertor methods. I have just finished an implementation of this here:
http://www.netbeans.org/source/browse/contrib/ttv/src/org/netbeans/swing...
Of course you will do better as you know a hell lot more about JTable than myself. What a pity that we had to implement this ourselves.

If you do it differently for columns and rows you will increase the API users confusion. Please keep the API same for both columns and rows - there is no conceptual difference between the 2 IMHO. You might argue about the usual number of columns versus rows but the optimalization arguments are weaker then to have clean API.

Best regards, David Strupl

herkules
Offline
Joined: 2003-06-12
Points: 0

IMHO there is a slight conceptual difference.

Columns have names, often types/classes and the number of columns is often pre-determined.

The number of rows very often isn't even limited (at least you cannot see the other end). The true number of row that have to be encountered for sorting/filtering can (but doesn't have to) be much bigger than those currently held be a TableModel.

dstrupl
Offline
Joined: 2003-06-11
Points: 0

I respectully disagree. Table is a grid - I can imagine an application where the number of columns is even not known beforehand. I agree that it is conventionally not used that way. But there is no conceptual difference. Ever thought of a flip function for a table (display rows as columns and vice versa)? It is quite hard to implement with JTable - or you have to provide your Look and Feel that can allow it.

But you are right that my arguments are not relevant here since the table has already hardcoded rows/columns difference and there is no simple way out. I thought there was a way - but the current column implementation is probably unusable for rows.

jidesoftware
Offline
Joined: 2004-10-07
Points: 0

Hi Scott,

JIDE has a SortableTableModel and FilterableTableModel for quite a long time. You can run webstart demo at http://www.jidesoft.com/products/1.4/jide_demo.jnlp. It has all our demos. The demo I want to show you is one called LargeSortableTable demo. It's a multiple column sortable table. You can add rows or delete rows on fly while keeping columns sorting order. We customized the table model event so that it is very efficient. We also allow user to specify their own sorting algorithm. You can also select some rows, then do sorting. You will notice all selections are kept.

We implemented SortableTableModel and FilterableTableModel using TableModelWrapper concept. It's basically a table model wraps another table model. You have any nested levels of wrappers. That's how we can support Sortable and Filterable table. SortableTableModel wraps FilterableTableModel then wraps the actual TableModel. I guess you did the same thing in your implementation. However this leads to the viewToModel() method you said. There is only one view but could have several nested table models. viewToModel() need to specify to which model. By default, it always default to the inner most table model. However it should have an option to get the row of any nested table model. I didn't see the actual API so I am not sure if you considered.

Anyway, let me know if you'd like to see some of our source code. Once Mustang releases sortable table, no one will use ours :-( so it would be better to share the code with you so that you don't need to waste time working on things we already addressed.

Cheer!

robjkc
Offline
Joined: 2004-03-22
Points: 0

I created a sorting mechanism using reflection like so:
BTW-If I don't have to getTableHeader().setReorderingAllowed(true); then sorting doesn't work. Anyone know why?

-------------------
public interface SortTableModel extends TableModel {
public boolean isSortable(int col);
public void sortColumn(int col, boolean ascending);
}
-------------------
public interface BeanMethods {
String[] getMethodNames();
}
---------------------
public class CustomerTableModel extends ListTableModel implements SortTableModel, BeanMethods {

private String[] columnNames = new String[]{"Cusomter Name", "Address", "Phone", "City", "State", "Zip"};

public Class[] columnTypes = {String.class, String.class, String.class, String.class, String.class, String.class};
private String[] methodNames = new String[]{"name", "addr1", "phone", "city", "state", "zip"};

public CustomerTableModel() {
}

public String[] getMethodNames() {
return methodNames;
}
....
public boolean isSortable(int col) {
return getMethodNames()[col].length() > 0;
}

public void sortColumn(int col, boolean ascending) {
Collections.sort(getList(), new BeanComparator(col, ascending, this));
}
}
--------------------
public class BeanComparator implements java.util.Comparator {
private int itsColumn;
private boolean _isAscending;
private BeanMethods itsMethods;

public BeanComparator(int col, boolean ascending, BeanMethods m) {
itsMethods = m;
itsColumn = col;
_isAscending = ascending;
}

public int compare(Object o1, Object o2) {
Object obj1;
Object obj2;
if (_isAscending) {
obj1 = o1;
obj2 = o2;
}
else {
obj2 = o1;
obj1 = o2;
}
String[] methodNames = itsMethods.getMethodNames();
try {
Object value1 = BeanUtil.getObjectProperty(obj1, methodNames[itsColumn]);
Object value2 = BeanUtil.getObjectProperty(obj2, methodNames[itsColumn]);

if (value1 instanceof Comparable && value2 instanceof Comparable) {
return((Comparable)value1).compareTo((Comparable)value2);
}
return 0;
}
catch (Exception e) {
throw new ExceptionAdapter(e);
}
}
}
----------------
public class JSortTable extends JTable implements MouseListener {

public JSortTable(SortTableModel tableModel) {
...
}

public void mouseReleased(MouseEvent event) {
TableColumnModel colModel = getColumnModel();
int index = colModel.getColumnIndexAtX(event.getX());
if (index < 0) return;
int modelIndex = colModel.getColumn(index).getModelIndex();

SortTableModel model = (SortTableModel)getModel();
if (model.isSortable(modelIndex)) {
// toggle ascension, if already sorted
if (itsSortedColumnIndex == index) {
_isSortedColumnAscending = !_isSortedColumnAscending;
}
else {
_isSortedColumnAscending = true;
if (event.isShiftDown()) {
_isSortedColumnAscending = !_isSortedColumnAscending;
}
}

itsSortedColumnIndex = index;
model.sortColumn(modelIndex, _isSortedColumnAscending);
}
}
}

zfqjava
Offline
Joined: 2003-09-22
Points: 0

Personally, I prefer the first approach.

If the sorting feature can be separated so that implement
this feature is freedom for developers, that's better.

In the first approach, I agree you think the selection index need to convert, that will add complex for developers, but from my experience for develop the different applications, when we use the JTable sorting feature, we often handle the selection by select it's value, not it's index, for example, if you create a file table model, you need to select the file object, not the index. So we get the selected value object in view (TableModel) or in model(TableModel) can use the same way.

jessewilson
Offline
Joined: 2003-06-14
Points: 0

Is there any plans to unify Swing models with the Collections framework? As I've mentioned before on the JDNC lists, I'm willing to contribute code from Glazed Lists to better the Java platform.

There advantages to doing sorting on lists of POJOs rather than on TableModels

- Share sorting logic between JTables, JList, JComboBoxes. In the future this could be extended to JTreeTables.

- By using POJOs, users can write Comparators that compare in the context of an entire row rather than a single cell. For example, in TableSorter.java there is no way to automatically break ties in sorting one column by using another column.

- Many of the index translation trouble can be removed. Instead of converting from view row index to model index to the actual Object, a collections-based implementation could have methods that return POJOs directly.[code] public Object getSelectedElement()
public List getSelectedElements()[/code]

The biggest limitation of this approach is that it is not compatible with the current TableModel-based approach. But I argue that it provides a much simpler programming model and implementing this approach will make a Swing developer's life easier.

-Jesse

elizarov
Offline
Joined: 2005-02-08
Points: 0

Jesse,

> Is there any plans to unify Swing models with the
> Collections framework?

This is an off-topic, but how you envision this integration? I have to
add to this, that I am also a great fan of this approach, because we
use a very similar one in our applications -- rows are just objects
returned from the "observable list" and each column knows how to
extract the corresponding value from the object (though, we are forced
to keep them in our separate xxxColumnModel and hack TableColumnModel
a lot). However, you can readily adapt TableModel interface to the
"glazed lists" approach (that is exactly what you actually do in your
EventTableModel class), but you cannot easily do the converse. That
is, some people need TableModel's approach where each cell at the
(row, column) coordinates is completely independent of the other cells
and they don't have any "row is an object" relationship. JTable's
approach is more flexible and, most importantly, does not prevent you
from constraining it to suite your applicatoin's particular needs.

In short, glazed lists is a nice additional package for certain
applications that use Swing, but I not see why it needs to be
integrated in Swing and in what way.

jessewilson
Offline
Joined: 2003-06-14
Points: 0

> ...That is, some people need TableModel's approach where each
> cell at the (row, column) coordinates is completely independent
> of the other cells and they don't have any "row is an object"
> relationship. JTable's approach is more flexible and, most
> importantly, does not prevent you from constraining it to suite
> your applicatoin's particular needs.

I completely agree that an arbitrary x/y grid is a useful and that the row-as-object approach is a smaller problem. Although JTable can be used as an arbitrary grid, much of its design lends itself to preferring rows as objects and columns as fields:
- JTable has a column header row but no row header
- TableModel has getColumnClass() but no getRowClass()
- TableModelEvent can specify first and last row, but only one column

I think that row-as-object is the only place where out-of-the-box sorting and filtering are appropriate. It is difficult to provide out-of-the-box sorting and filtering for arbitrary x/y grids.

Although it is not necessary to support sorting by hidden fields, this is a must-have for filtering. For example, filtering emails may match against the message bodies, even if they is not displayed in a column. It would be awkward apply sorting to the TableModel but filtering in a different type of model.

mossprescott
Offline
Joined: 2008-04-22
Points: 0

I'm a little surprised that you seem to be focused on what convention to adopt for row indexes. Hopefully that's just because all the bigger issues are already solved and you're just not telling us!

Echoing some of Jesse Wilson's comments, I think what's most important is that the implementation provide hooks for developers to get the right look and feel, while leaving them the freedom to implement sorting in their models as they see fit.

There are just too many possible interpretations of sorting (by one column, more than one column, up/down vs. whatever else, special rows that sort differently, etc.) for any one implementation to cover them all. And there are many more approaches to sorting a model than could ever be covered by one builtin implementation.

But there _is_ a great need for the look and feel to handle the machanics of selecting the column to sort by and drawing the sort indicator(s). So I guess my vote is for a nice, rich API to control column selection and sort indicators, but an interaction with the model that allows for sorting to implemented by each model writer as needed.

Of course, I wouldn't object to a nice, simple, default sortable model included with Swing...

All those of us who have used JTable in real non-trivial applications can testify that the default renderers and models are far too simple to handle more than the trivial examples; please make sure the implementation you come up with doesn't make it even harder to write these more sophisticated custom pieces (see the earlier remarks about TableColumnModel).

- Moss

PS. what I'd _really_ like to see is a more ambitious effort to come up with a more usable TableModel/JTable/JTableHeader/TableColumn interaction. The current scheme is a good simple foundation with an overly constraining model/view interaction. What do people think; can we wait for Sun, or should there be new efforts to do something better?

herkules
Offline
Joined: 2003-06-12
Points: 0

The problems I encountered with table sorting in the company I am forced to work for:

I implemented it with a TableModel decorator that maintains an index over the decorated model.

The advantage is that an arbitrary TableModel can be sorted with that, bc. sorting takes place in the decorator which is hidden in the widget XXXTable extends JTable.

In practise, sorting and filtering cannot always be performed on the data that *currently* is contained in the tablemodel. We have TableModels that load further data whenever the last row touched thus providing a TableModel-window over a larger dataset that cannot be loaded completely from server to client.
In this case, it is mandatory that sorting/filtering can take place on the server!

Thus, the pure decorator approach didn't hold.

Additionally, the decorator produces two kinds of coordinates. View coordinates and model coordinates. It is very hard to keep them separated. Lots of issues on this side! In fact, thinking of column indeces, this dilemma exists today already. Symmetrie in column and row reordering would be nice in this respect anyway.

Making the sorting/filtering part of the TableModel interface IMHO is not a particularly good idea, bc. is (a) makes implementation of new models *much* harder and (b) might be too restricted for all cases of reordering that people will think of.

What about a set of model interfaces that JTable understands?

Scott Violet

Well, I'm glad to hear someone say they don't like having to deal with
all this model to view translation.

Regardless of which approach we go with you'll have the option of
implementing your own sorting algorithm. Be that at the model level or
in the database.

-Scott

carloscs
Offline
Joined: 2003-08-11
Points: 0

The first approach [mantain a row model + viewToModel/modelToView methods].

The second approach only changes one set of problems for another, and (at least to me) seems much more intrusive (more changes to existing code in jtable) for no added benefit.

The view approach also has the advantage that it's already used in the column model, so it shouldn't be too hard for developers to get acquainted to the changes.

By the way, I would suggest that any ui aspects of sorting/filtering (such as which comparators to use, how to specify filters, etc) be discussed in a new thread, leaving this thread only to the model/view problems, sort alghorithms, etc

Scott Violet

swing-feedback@javadesktop.org wrote:
> The first approach [mantain a row model + viewToModel/modelToView methods].
>
> The second approach only changes one set of problems for another, and
> (at least to me) seems much more intrusive (more changes to existing
> code in jtable) for no added benefit.

I agree that it is a huge change for JTable, but I think it means less
changes for developers. I tend to think most developers want to operate
in model coordinates and so the second approach addresses that.

-Scott

MCarlin
Offline
Joined: 2006-02-17
Points: 0

Scott:
Will you be starting a separate thread regarding table filtering?

I second Jesse's excellent suggestions. In addition to having the table maintain the selection when the sort order is changed, there should be a way to have the table automatically keep the selection in view after sorting. Regarding allowing multiple comparators per column, there should be an easy way to dynamically add or remove comparators to a column (this would allow a user to create his own custom sorters and apply them at run time). You should be able to chain together a subset of the comparators to support primary sort criteria, secondary sort criteria, tertiary, etc. within a column.

I second Keith's call for an API to support finding visual positions of elements in a table.

MC

Scott Violet

Michael,

Filtering will be done at the same time. The questions I originally
raised equally apply to filtering.

Thanks for the feedback!

-Scott

MCarlin
Offline
Joined: 2006-02-17
Points: 0

To support table filtering, the ability to display some sort of selection widget in the table header allowing the user to select filter criteria for a column would be useful. Perhaps something similar to the way Excel supports filtering on a column or perhaps something like the examples at the sites below.

Perhaps something like Figure 3 at the following site:
http://www-106.ibm.com/developerworks/web/library/us-reduce/index.html

Perhaps something like the FilterTable screenshot at the following site:
http://webpages.charter.net/daltontk/components.html

MC

MCarlin
Offline
Joined: 2006-02-17
Points: 0

Following up on my own message, when the table is filtered, the row selection should be preserved if that row is still present in the table after filtering. If the selected row is no longer present after filtering, some sort of event should be sent to interested listeners. This would allow one to clear a "details" view if the selected row is no longer present in a "master" view.

MC

snpe
Offline
Joined: 2003-09-16
Points: 0

I have next model :
- list from database - database can be > million rows, but I cache xxx and call from database when go out of range
- database do sorting, filtering etc (i make call to database with hibernate hql)
I need :
- good sort indicator for any column headers (possible multiple)
- features that add components between rows (for hierarchy structure - this component is another jtable for me - detail view)

regards
Haris Peco

elizarov
Offline
Joined: 2005-02-08
Points: 0

There is one most important thing to note. Whatever you do, you are
not going to suit every application need. For example, in our
applications we have extended JTable to represent, as we call it,
"NodeTable", where each "node" occupies multiple rows in the table but
represents a single conceptual entity for the client (additional rows
show multiple details lines per entity). This multi-line entity is
selected in table as a whole (with all its rows). Moreover, we sort
not individual rows, but those multi-line entities. I would not expect
anything like this to be in Swing, because hardly anyone else needs
it.

Therefore, the most important design decision is to make it as
flexible and as extensible as possible. By this I mean:

* Make sure that there is a separate "SortingModel" interface or
something like this, which can be replaced by anyone and the default
implementation is just what it is -- a default implementation.

* Make sure that view itself (JTable/TableUI) consults with
"SortingModel" to decide when to show sorting indicators in the column
headers. If I need to have my own sorting implementation I still would
like to have access to the 'native' (that is, provided by Swing) way
to show sorting indicators in column headers.

I cannot stress this enough! Strive for the kind of separation that
exists between JTable and TableModel, between JTable and
ListSelectionModel. Please, do not make the kind of mistake that they
did with TableColumnModel, because it does very wrong things. JTable,
instead of treating ColumnModel as something external to modify and
listen for, goes and uses ColumnModel as something it owns
exclusively. Therefore, you cannot just share two column models
between tables, which is an awfully unfortunate.

So note, that for a SortingModel, the first "extensibility test" I am
going to make is to check if I am able share it between multiple
tables even when those tables have different data items inside.

I completely agree with the post about a need to explicitly have
Comparators. Moreover, you should not assume that reverse sorting
direction is made by just reversing normal order (though, it shall be
a reasonable default). Some applications has special requirements like
always placing certain items at the top/bottom regardless of the
sorting order.

Moreover, in some applications I would like to take into account
additional columns when sorting, even though a user has just opted to
sort by a single column. For example, if user chooses to sort emails
by sender, then application may still be required by design to
additionally sort emails with the same sender by their time. I am not
sure that default implementations shall readily give this kind of
flexibility, but I must be able to drop in my own replacement
implementations to do the trick. All this means, that on some level
there should be an overrideable method with a signature similar to the
following one:

public int compare(int row1, int row2, TableModel model);

I do not want to stress performance, but I still want to mention it
here. If I have a performance-critical code and I find the sorting to
be on a critical path, then I'll write a replacement implementation.
Though, the latter is unlikely to happen, because it should actually
do only a fraction of the work that "paint" does (the CPU spent on
sorting work shall be overshadowed by painting). But here is a catch
-- only a visible part of table is painted, but everything is sorted,
even invisible things. And an invisible part can contain a much, much
more rows than visible one. Thus, performance can turn out to be a
problem in quite unexpected places.

A garbage-less implementation of sorting algorithm will be a bonus
(though not a requirement, as I pointed before). That is, do not use
Collectoins/Arrays.sort... To have a stable-sort guarantee write your
own mergesort algorithm and re-use the second array across sortings,
or just use a specialized version of quicksort that takes into account
original indices of rows to figure out the order when comparator
returns zero (you should keep/know those indices somewhere anyway).

A stable sorting is a nice feature to have, even though it can be
always emulated if is does not present by original design. As an
added benefit, when you a have a stable sorting, you implement a
"non-sorted" mode by simply returning zero on any call to 'compare'
method.

Now on the question of the original poster about indices (model
indices vs view indices). Just to preserve the spirit of existing
table sorting frameworks that typically work on the TableModel layer,
I would suggest to do the same. That is, and a sorting layer somewhere
in between TableModel and JTable itself and leave all methods in
JTable to operate on _view_ indices.

That is, setRowSelectionInterval(0,0) shall select first row in the
view. However, getSelectionModel().setSelectionInterval(0,0) shall
select first row in the model, because ListSelectionModel shall not
change on reordering of items, even though view shall still show
selection rectangle on the view rows with the same model rows. I mean,
visually selection will change to track the reordering of the
corresponding rows from the model, even though selection will not
change from the ListSelectionModel's point of view.

Scott Violet

Roman,

Thanks for the thorough and detailed response.

One point I wanted to comment on is:

> Now on the question of the original poster about indices (model
> indices vs view indices). Just to preserve the spirit of existing
> table sorting frameworks that typically work on the TableModel layer,
> I would suggest to do the same. That is, and a sorting layer somewhere
> in between TableModel and JTable itself and leave all methods in
> JTable to operate on _view_ indices.

The amount of changes required to do otherwise is very large, and
requires changing the UIs. I believe this is why most frameworks take
the approach of decorating the model and not changing JTable itself.

> That is, setRowSelectionInterval(0,0) shall select first row in the
> view. However, getSelectionModel().setSelectionInterval(0,0) shall
> select first row in the model, because ListSelectionModel shall not
> change on reordering of items, even though view shall still show
> selection rectangle on the view rows with the same model rows. I mean,
> visually selection will change to track the reordering of the
> corresponding rows from the model, even though selection will not
> change from the ListSelectionModel's point of view.

If we go the route of keeping coordinates in terms of the view than why
wouldn't the selection model be in view coordinates too? With this
approach a sort would end up changing the contents in terms of the view,
but in terms of the model it would be the same.

-Scott

elizarov
Offline
Joined: 2005-02-08
Points: 0

Scott,

here are some additional clarifications to my opinion.

> The amount of changes required to do otherwise is very large, and
> requires changing the UIs. I believe this is why most frameworks
> take the approach of decorating the model and not changing JTable
> itself.

I believe that is correct, too. That one of the reasons I believe it
is how it shall be done in JTable. Anybody who ever did sorting will
immediately recognize this approach as a natural one. On the other
side, all novice users (who never did sorting before) will have to
study manual/tutorial on that topic anyway and will easily learn any
approach you choose to implement.

Carlos had also made a great point about the fact, that there already
_is_ an approach to coordinates problem that is taken by
TableColumnModel. That is, JTable.getValueAt(row, column) treats
column argument in the view's display order, and not in the
TableModel's column order. Treating row argument the other way around
just adds a lots of confusion to the picture.

> If we go the route of keeping coordinates in terms of the view than
> why wouldn't the selection model be in view coordinates too? With
> this approach a sort would end up changing the contents in terms of
> the view, but in terms of the model it would be the same.

It is mostly a matter of aesthetics. ListSelectionModel is, by its
name, a model and shall ideally work in the model coordinates. It is
also aesthetically pleasing to know that all JTable's subordinate
models (TableColumnModel/TableModel/ListSelectionMode/SortingModel?)
are independent in the sense that each of them changes independently
of the other. Ideally, models shall change only as a result of direct
user action, or as a result direct change of the data underlying the
model. They should not change as a side-effect of the change in some
other model.

However, there is also another, non aesthetics, but use case based
reason. We have the following use case in our applications. There are
two tables side-by-side that share TableModels and
ListSelectionModels, but have different columns and different sorting
models. When user selects first row in the left table, for example,
then the corresponding row in the right table must be highlighted also
and it may not be the first one, due to the different sorting. Keeping
ListSelectionModel in model's coordinates automatically addresses this
use case.

Does anybody have an application with similar two tables (same data,
different sorting) but where selection of the first row on the left
also selects the first row on the right? I have seen none and I doubt
this makes any sense.

I agree, that keeping ListSelectionModel in the model coordinates
presents a number of technical issue to overcome, such as multiple
model changes on a simple user action of selection extension. But I
believe the benefits are worth the effort.

MCarlin has also raised a question on filtering. My take on that is to
keep a row selected in a ListSelectionModel even if this row was
filtered away. However, applications want to perform some action upon
visible (i.e., not filtered away) and selected rows. Thus, it is
completely logical that JTable.getSelectedColumns shall report only
such rows, that is, it shall not include rows that are selected in
ListSelectionModel, but are not accepted by the filter.

Anyway, you cannot just let yourself just go and change
ListSelectionModel every time filter changes. You know that I can use
JTable just as a kind of non-interactive view, and my particular
ListSelectionModel may be backed by a read-only data-structure that
simply reports, for example, that all records with a particular
pattern inside are selected. Why shall it support modification in a
view-only/read-only mode? So, there is an MVC mantra for you to keep
in mind: "If there is no user interaction happening, then models must
not be changed by the controller even if some other models change".

However, I cannot consider myself an expert on filtering use cases,
because we never do it in our applications on the UI side in a
user-controlled way. We only do fixed filtering to split a list of
items into multiple tables by some fixed criteria. We take data from
the large databases and it always forces us to do all the real
filtering on the DB side via SQL "SELECT ... WHERE ...". On the other
hand, when number of data items is so small that it is practical to
load them all into the memory, then UI-side filtering also does not
seem important for me, because it becomes convenient just to sort them
all and quickly lookup the item you need in the alphabetical order. I
hope that somebody with an experience in real-life application with
rich UI-side filtering will share with us the use cases that are
encountered there and, importantly, how it interacts with selection
from the user perspective.

Scott Violet

Roman,

swing-feedback@javadesktop.org wrote:
> Scott,
>
> here are some additional clarifications to my opinion.
>
>
>>The amount of changes required to do otherwise is very large, and
>>requires changing the UIs. I believe this is why most frameworks
>>take the approach of decorating the model and not changing JTable
>>itself.
>
>
> I believe that is correct, too. That one of the reasons I believe it
> is how it shall be done in JTable. Anybody who ever did sorting will
> immediately recognize this approach as a natural one. On the other
> side, all novice users (who never did sorting before) will have to
> study manual/tutorial on that topic anyway and will easily learn any
> approach you choose to implement.
>
> Carlos had also made a great point about the fact, that there already
> _is_ an approach to coordinates problem that is taken by
> TableColumnModel. That is, JTable.getValueAt(row, column) treats
> column argument in the view's display order, and not in the
> TableModel's column order. Treating row argument the other way around
> just adds a lots of confusion to the picture.

This adds no end of confusion, even for us... It's so easy to forget to
do a conversion.

>>If we go the route of keeping coordinates in terms of the view than
>>why wouldn't the selection model be in view coordinates too? With
>>this approach a sort would end up changing the contents in terms of
>>the view, but in terms of the model it would be the same.
>
>
> It is mostly a matter of aesthetics. ListSelectionModel is, by its
> name, a model and shall ideally work in the model coordinates.

That really depends. The related view classes most definitely want the
coordinates in terms of the view. And when developer say single
selection interval I tend to think that's in terms of the view. I'm not
positive on the last one though. What do folks think?

> It is
> also aesthetically pleasing to know that all JTable's subordinate
> models (TableColumnModel/TableModel/ListSelectionMode/SortingModel?)
> are independent in the sense that each of them changes independently
> of the other. Ideally, models shall change only as a result of direct
> user action, or as a result direct change of the data underlying the
> model. They should not change as a side-effect of the change in some
> other model.

That really depends upon the perspective. The selection model has
double duty, it's used by both the view classes and developer and it's
tricky to balance both out.

> However, there is also another, non aesthetics, but use case based
> reason. We have the following use case in our applications. There are
> two tables side-by-side that share TableModels and
> ListSelectionModels, but have different columns and different sorting
> models. When user selects first row in the left table, for example,
> then the corresponding row in the right table must be highlighted also
> and it may not be the first one, due to the different sorting. Keeping
> ListSelectionModel in model's coordinates automatically addresses this
> use case.
> Does anybody have an application with similar two tables (same data,
> different sorting) but where selection of the first row on the left
> also selects the first row on the right? I have seen none and I doubt
> this makes any sense.
>
> I agree, that keeping ListSelectionModel in the model coordinates
> presents a number of technical issue to overcome, such as multiple
> model changes on a simple user action of selection extension. But I
> believe the benefits are worth the effort.

I'm bothered by yet more inconsistency. If we go the wrapping approach
and have the selection in terms of the model then some JTable methods
will be in terms of the view, and some in terms of the model. For
example, you're arguing the selection methods should remain in terms of
the model, but what about the rest that have row arguments:
getValueAt(int,int), setRowHeight(int,int). They have to remain in
terms of the view and so suddenly it's pretty tricky to remember which
one you're dealing with. All row based methods should be in terms of
the model, or in terms of the view.

Consider getSelectedColumns, it's in terms of the view.

> MCarlin has also raised a question on filtering. My take on that is to
> keep a row selected in a ListSelectionModel even if this row was
> filtered away. However, applications want to perform some action upon
> visible (i.e., not filtered away) and selected rows. Thus, it is
> completely logical that JTable.getSelectedColumns shall report only
> such rows, that is, it shall not include rows that are selected in
> ListSelectionModel, but are not accepted by the filter.
>
> Anyway, you cannot just let yourself just go and change
> ListSelectionModel every time filter changes. You know that I can use
> JTable just as a kind of non-interactive view, and my particular
> ListSelectionModel may be backed by a read-only data-structure that
> simply reports, for example, that all records with a particular
> pattern inside are selected. Why shall it support modification in a
> view-only/read-only mode? So, there is an MVC mantra for you to keep
> in mind: "If there is no user interaction happening, then models must
> not be changed by the controller even if some other models change".

Your argument holds if the selection is in terms of the model, not if it
is terms of the view.

One option is to introduce another selection model that is in terms of
the model and keep JTable's current selection model in terms of the
View. On filter/sort the model one doesn't change, but the view one
does. Keeping two models in sync is always a pain, but this is
certainly possible.

Thanks for the feedback,

-Scott

keithkml
Offline
Joined: 2003-06-10
Points: 0

I think sorting should be done independently of the model - that is, 100% by the view, just like column reordering. JTable should support a concept of virtual and physical indices for both rows and columns. I believe this could be made transparent to current code by automatically converting from view to model and vice versa in the necessary cases.

Then, I think JTable needs a new API for controlling and monitoring sorting, as well as finding visual positions of elements and vice versa.

robmv
Offline
Joined: 2004-10-30
Points: 0

Our working implementation has something like this

interface SortableTableModel extends TableModel {
...
boolean isColumnSortable(int column);
void sort(int column, int direction);
void unsort();
...
}

This interface is needed for models that can be sorted

interface SortableTableModelListener {
void columnSortChanged(SortableTableModelEvent event);
}

Used to signal when the model is changing its sort state

class BasicSortableTableModel implements SortableTableModel {
...
void setColumnSortable(int column, boolean value) ...
int getUnsortedIndex(int index) ...
void sort(int column, int direction) ...
unsort() ...
...
}

And provide a default implementation of SortableTableModel that wraps an existing TableModel with default implementations (sort by text, by numerical order based on the column class)

I think that the main problem that will exist with any implemetation is the interpretation of the index of the rows. We choosed to interpret them as the visual position, and each time we need to get the real row we need to call [i]getUnsortedIndex[/i]. Two reasons led us to this approach.

- Not able to change the implementation of JTable
- Let the developers do things as normally do, for example tell the table to scroll to a row index (visual row index)

We let the model manage the sorting, that way we have self ordering models. For example, when the user finish editing a cell the model implementation can choose to resort itself, or move the editing row latest position, or stay where it is

I hope this helps

Scott Violet

Robert,

Thanks for the feedback.

> I think that the main problem that will exist with any implemetation is
> the interpretation of the index of the rows. We choosed to interpret
> them as the visual position, and each time we need to get the real row
> we need to call [i]getUnsortedIndex[/i]. Two reasons led us to this
> approach.
>
> - Not able to change the implementation of JTable

That's why I'm asking. We can change JTable to have coodinates remain
in terms of the view.

> - Let the developers do things as normally do, for example tell the
> table to scroll to a row index (visual row index)

How often do you find yourself doing this?

If we go the route of having coordinates remain in terms of the view
developers will have to change their code. On the other hand if we keep
coordinates in terms of the model developers may not have to change
their code.

Thanks!

-Scott

robmv
Offline
Joined: 2004-10-30
Points: 0

> > - Let the developers do things as normally do, for
> example tell the
> > table to scroll to a row index (visual row index)
>
> How often do you find yourself doing this?

Well I do that each time the user interact with my applications, and that results on adding an row to the table, as the table is sortered, we guide to the user to the exact position where the row was added.

>
> If we go the route of having coordinates remain in
> terms of the view
> developers will have to change their code. On the
> other hand if we keep
> coordinates in terms of the model developers may not
> have to change
> their code.
>
> Thanks!

I think that the best option is to allow the table ordering optional, that will allow compatibility with any written code, as ordered coordinates and unordere coordinates will be the same. Problems will arise when developers start enabling the table ordering

If the current coordinate system interpreted as relative to the model, some problems can arise for code that assume things that they must not assume. for example with SINGLE_INTERVAL_SELECTION: the view select rows 2, 3 and 4 but the model tells that the selection is 1 5 and 8. some code must be assuming that as the selection is a single interval, with the first index and the last index is enough, and if the developer wrote code to remove selected items, in this example it will remove from 1 to 8

>
> -Scott

jessewilson
Offline
Joined: 2003-06-14
Points: 0

There are many apects of table sorting that should be accessible to the programmer:

- The list of Comparators for each column. Normally this is just Comparable and its reverse. But there are situations where it is useful to have multiple Comparators for a single column. For example, an email address column should be sortable either by user name or by domain name. This would be implemented with two comparators for that column. Note that storing Comparators in a List provides a natural representation of an unsortable column: an empty List.

- Whether sorting is performed live or in batch. For example, if a table is sorted alphabetically, it can be distracting when a row moves after it has been edited. Similarly this behaviour may be desirable, particularly for uneditable tables. In an email application, new messages should arrive in 'sorted order' rather than at the table's end. Compare Mac OS X Finder (live sort) to Windows Explorer (batch sort) for examples.

- The table should maintain the selection when the sort order is changed. This is impossible to do with the sort strategy "SINGLE_INTERVAL"

- Direction indicator icons. Appropriate icons should be used automatically for the current look and feel. It should also be possible to specify a custom icon set. It would be nice to support icons from image files plus icons created in code via Java2D.

- How to sort by multiple columns simultaneously. This is a challenging interaction to make a simple user interface for. When a user clicks on a column's header, their intentions can vary:
- clear the Comparator on the clicked column,
- reverse the Comparator on the clicked column
- move to the new Comparator for the clicked column
- sort by the clicked column only
- sort by the clicked column to break ties from other sorting columns

In order to make all of this functionality accessible to the Java programmer, I recommend not hiding how sorting is implemented inside the JTable class.

A year or so ago I wrote a document comparing three implementations of table sorting for Swing. It is important to note the performance differences between TableSorter, JDNC and Glazed Lists for large (>1000) data sets:
http://publicobject.com/glazedlists/sortedjtables/

This is a very important issue to me since I've spent the last 18 months working on table sorting and related issues with my Glazed Lists project.

Cheers,
Jesse

Scott Violet

Jesse,

Thanks for the most excellent feedback.
All the aspects you have outlined are planned for.

> In order to make all of this functionality accessible to the Java
> programmer, I recommend not hiding how sorting is implemented inside the
> JTable class.

I don't think either approach limits the functionality that can be
exposed. By that I mean that if we change JTable to keep all
coordinates in terms of the model we could still expose all the features
you have outlined.

-Scott