Skip to main content

Functors & JDNC

36 replies [Last post]
rbair
Offline
Joined: 2003-07-08
Points: 0

This post is mostly directed to Dave Hall (hey, how's it goin'?), but anybody else that wants to pipe up, I'm looking forward to hearing from you too!

There are several places in JDNC where having a functor might be useful (filtering, sorting, etc). In the latest DataSet API code in the incubator (reorg, org.jdesktop.dataset) there are DataColumns and DataValues. I don't really want to get into the architecture of the code right now (a full writeup will be forthcoming), but suffice it to say that I could have 3 DataColumns: Price, Quantity, ExtPrice, where Price and Quantity contain values from the underlying data store, and ExtPrice is Price * Quantity.

What I want is a simple parser & expression syntax so that if the DataColumn has expression="Price * Quantity" the parser can figure out that it needs to find the DataColumn Price and the DataColumn Quantity and for each row compute its value. Likewise, the DataValue could contain the expression "sum(ExtPrice)" which would sum the values at each row for the DataColumn ExtPrice.

Is JGA what I want for this? If not, is there another expression parser library that could be used? I don't need anything too terribly complex (at some point it becomes easier to ask people to code in Java rather than use the expression syntax). Simple expressions are all I care about.

Thanks
Richard

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Richard Bair

>the DataTable.getValue(DataRow,DataColumn) method is returning strings from
>the price column, even though it looks like they should be returning
>BigDecimals. Is there something in the metadata that we can change?

I found the real problem, the (hack) implementation that is reading data
from XML files isn't converting the datatype properly. Here's a (hack) way
to get around it for this demo:
[code]
public static void readXml(String xml) {
...
String text = colNode.getTextContent();
//convert the text to the appropriate object
of the appropriate type
Object val = text;
Class type =
table.getColumn(colNode.getNodeName()).getType();
if (type == BigDecimal.class) {
val = new BigDecimal(text);
}
//TODO do the other types
row.setValue(colNode.getNodeName(), val);
...
}
[/code]

A more robust implementation will be forthcoming, but this should solve the
problem of the "price" column not being a big decimal. Assuming that the
schema type is xs:decimal of course.

Richard

---------------------------------------------------------------------
To unsubscribe, e-mail: jdnc-unsubscribe@jdnc.dev.java.net
For additional commands, e-mail: jdnc-help@jdnc.dev.java.net

Richard Bair

Dave,

If I wanted to build everything here, what do I need? I assume that I just
need the latest jga jar and the diffs you sent?

Thanks
Richard

---------------------------------------------------------------------
To unsubscribe, e-mail: jdnc-unsubscribe@jdnc.dev.java.net
For additional commands, e-mail: jdnc-help@jdnc.dev.java.net

dhall
Offline
Joined: 2006-02-17
Points: 0

> If I wanted to build everything here, what do I need?
> I assume that I just
> need the latest jga jar and the diffs you sent?
>

I'm pretty sure that's all I've done in the main tree: I've just checked in a new jga jar in my incubator area, and I added a tax column to the demo table.

Here's the change in the .xsd file

[code] expression="price * (java.math.BigDecimal) 0.15" />
[/code]

I ended up leaving the type alone, using it to describe the result of the expression. I'm not attached to any of the XML syntax. I'm working on a couple more changes to the parser that'll eliminate the cast.

You had me going there for a while: I was feeling a little
sheepish that the fix might've turned out to be such a simple thing in the metadata. I thought I'd looked there, but it was late and today I wasn't so sure I had.

Dave

dhall
Offline
Joined: 2006-02-17
Points: 0

Richard:

After applying your hack^H^H^H^Hpatch to DataSet, I got to see the
first computed column: the tax column computes and displays correctly.
Of course, the tax is being displayed to 4 significant digits, but
it's a start.

I may have spoken a little too soon as to whether you have everything you
need to build it: you don't have the parser extension that knows how
to look up column names apropriately. I put this in DataSet.java, and
changed the declaration of the parser to use this new class.

[code]
import net.sf.jga.fn.UnaryFunctor;
import net.sf.jga.fn.adaptor.ApplyUnary;
import net.sf.jga.fn.adaptor.ConstantUnary;
import net.sf.jga.fn.adaptor.Identity;
import net.sf.jga.fn.property.InvokeMethod;
import net.sf.jga.parser.FunctorParser;
import net.sf.jga.parser.FunctorRef;
import net.sf.jga.parser.ParseException;
import net.sf.jga.parser.UnaryFunctorRef;

...

static private class ColumnParser extends FunctorParser {
protected FunctorRef checkReservedNames(String name) throws ParseException {
// Assuming (for now) that the bound object is a DataTable
DataTable tbl = (DataTable) super.getBoundObject();
DataColumn col = tbl.getColumn(name);
if (col != null) {
// Builds a functor that takes a Row, and returns an array consisting
// of the given row and the column we just found. The array will be
// passed to the method invocation logic.
ApplyUnary args =
new ApplyUnary(new UnaryFunctor[]
{ new Identity(),new ConstantUnary(col) });

// getValue(col,row) method as a functor
InvokeMethod getValue =
new InvokeMethod(DataTable.class,"getValue",
new Class[] {DataRow.class, DataColumn.class});
// tie the two together
UnaryFunctor value = getValue.bind1st(tbl).compose(args);
return new UnaryFunctorRef(value, DataRow.class, ARG_NAME[0], col.getType());
}

return null;
}
}
[/code]

The getBoundObject method is new, but I'm fairly sure that it is the last
jarfile that I checked into my incubator: if it isn't, I'll have to check
one in this evening (late afternoon on your coast: I can't access the CVS
tree from work). This code doesn't have to live inside DataSet.java:
it was the easiest place to drop it for now.

Please note that the parser is definitely not the place to look for examples
of typical functor usage: this code is a good bit scarier than I expect to
be the typical case for users of jga.

I have the import static logic that we talked about early in the thread
implemented in the base parser, leading toward support of summary operations
for the DataValue. I didn't see anything in the xsd file that describes how
the DataValue would be defined, and I think I'm missing the spot in the
java implementation where I need to look to drop in the necessary patch.

Dave Hall
http://jga.sf.net/
http://jroller.com/page/dhall/

dhall
Offline
Joined: 2006-02-17
Points: 0

Here's a set of patches to the existing jdnc code that make it work. I don't expect grammar changes to affect this, so I'll go ahead an post it here.
[code]
Index: swingx/src/java/org/jdesktop/dataset/DataColumn.java
===================================================================
RCS file: /cvs/jdnc/swingx/src/java/org/jdesktop/dataset/DataColumn.java,v
retrieving revision 1.1
diff -u -r1.1 DataColumn.java
--- swingx/src/java/org/jdesktop/dataset/DataColumn.java 23 Feb 2005 17:51:31 -0000 1.1
+++ swingx/src/java/org/jdesktop/dataset/DataColumn.java 10 Mar 2005 02:51:49 -0000
@@ -8,6 +8,7 @@
import java.beans.PropertyChangeListener;
import java.beans.PropertyChangeSupport;
import java.util.logging.Logger;
+import net.sf.jga.fn.UnaryFunctor;
import org.jdesktop.dataset.NameGenerator;

/**
@@ -74,6 +75,11 @@
* be the same, as determined by .equals()).
*/
private boolean keyColumn;
+
+ /**
+ * Expression to be evaluated for computed columns
+ */
+ private UnaryFunctor expression;

/**
* Create a new DataColumn. To construct a DataColumn, do not call
@@ -220,6 +226,20 @@
pcs.firePropertyChange("keyColumn", oldVal, value);
}
}
+
+
+ public UnaryFunctor getExpression() {
+ return expression;
+ }
+
+ public void setExpression(UnaryFunctor gen) {
+ if (gen != expression) {
+ UnaryFunctor oldGen = expression;
+ expression = gen;
+ pcs.firePropertyChange("expression", oldGen, gen);
+ }
+ }
+

public void addPropertyChangeListener(PropertyChangeListener listener) {
pcs.addPropertyChangeListener(listener);
@@ -236,4 +256,4 @@
public void removePropertyChangeListener(String propertyName, PropertyChangeListener listener) {
pcs.removePropertyChangeListener(propertyName, listener);
}
-}
\ No newline at end of file
+}
Index: swingx/src/java/org/jdesktop/dataset/DataRow.java
===================================================================
RCS file: /cvs/jdnc/swingx/src/java/org/jdesktop/dataset/DataRow.java,v
retrieving revision 1.1
diff -u -r1.1 DataRow.java
--- swingx/src/java/org/jdesktop/dataset/DataRow.java 23 Feb 2005 17:51:30 -0000 1.1
+++ swingx/src/java/org/jdesktop/dataset/DataRow.java 10 Mar 2005 02:51:49 -0000
@@ -11,6 +11,7 @@
import java.util.Collection;
import java.util.HashMap;
import java.util.Map;
+import net.sf.jga.fn.UnaryFunctor;

/**
@@ -85,6 +86,18 @@
}

public Object getValue(DataColumn col) {
+ UnaryFunctor expression = col.getExpression();
+ if (expression != null) {
+ try{
+ return expression.fn(this);
+ }
+ catch (Exception x) {
+ // TODO: fix the error handling
+ x.printStackTrace();
+ }
+
+ }
+
return cells.get(col).value;
}

@@ -154,4 +167,4 @@
}
}
}
-}
\ No newline at end of file
+}
Index: swingx/src/java/org/jdesktop/dataset/DataSet.java
===================================================================
RCS file: /cvs/jdnc/swingx/src/java/org/jdesktop/dataset/DataSet.java,v
retrieving revision 1.3
diff -u -r1.3 DataSet.java
--- swingx/src/java/org/jdesktop/dataset/DataSet.java 25 Feb 2005 19:46:13 -0000 1.3
+++ swingx/src/java/org/jdesktop/dataset/DataSet.java 10 Mar 2005 02:51:50 -0000
@@ -24,6 +24,7 @@
import javax.xml.xpath.XPath;
import javax.xml.xpath.XPathConstants;
import javax.xml.xpath.XPathFactory;
+import net.sf.jga.parser.FunctorParser;
import org.jdesktop.dataset.event.TableChangeEvent;
import org.w3c.dom.Document;
import org.w3c.dom.Node;
@@ -239,6 +240,10 @@
*/
public static DataSet createFromSchema(String schema) {
DataSet ds = new DataSet();
+
+ //set up a functor parser to build expressions
+ FunctorParser parser = new FunctorParser();
+
//set up an XML parser to parse the schema
try {
DocumentBuilder db = DocumentBuilderFactory.newInstance().newDocumentBuilder();
@@ -260,6 +265,10 @@
Node tableNode = tableNodes.item(i);
DataTable table = ds.createTable();
table.setName(xpath.evaluate("@name", tableNode));
+
+ // hook the parser to the table, temporarily (while the table is being defined)
+ parser.bindThis(table);
+
//construct the table's data columns
NodeList columnNodes = (NodeList)xpath.evaluate("complexType/sequence/element", tableNode, XPathConstants.NODESET);
for (int j=0; j @@ -306,7 +315,11 @@
} else {
System.err.println("unexpected classType: '" + classType + "'");
}
- }
+
+ String exp = xpath.evaluate("@expression", colNode);
+ if ( ! ("".equals(exp))) {
+ col.setExpression(parser.parseUnary(exp, DataRow.class));
+ } }
}
}
} catch (Exception e) {
@@ -524,4 +537,4 @@

System.out.println(ds.getSchema());
}
-}
\ No newline at end of file
+}
[/code]

rbair
Offline
Joined: 2003-07-08
Points: 0

> It looks like the likely place to start will be in
> the adventure/resources/dataset.xsd file. I thought
> I'd add a
>
> > minOccurs="0" default="price * 0.10"/>

Minor nit: I'm just getting up to speed on xml schema myself, but I think you'd want the type to be something like "jdnc:expression" instead of "xs:expression" because the latter implies that there is an expression type in the xmlschema namespace. Maybe change the attribute name "default" to "expression" or something too. Not really important, we can work the specifics of that out later.

> If I'm offbase here, is there a better place to
> start? It looks from this code in MainWindow.java
>
> [code]
> TabularDataModelAdapter activitiesDetailModel = new
> TabularDataModelAdapter(ds.getTable("activitiesDetail"
> ));
> new DirectTableBinding(activitiesTBL,
> activitiesDetailModel, new String[]{"name",
> "description", "price" });
> [/code]
>
> like what I want to do is add "discount" to the
> column list and somehow get "price * 0.10" into the
> meta data.

At this point, you'll need to hack up the DataSet's static createFromSchema(String xml) method to include a check on DataColumns for the "jdnc:expression" attribute. If you find it, set the expression on the column. So, something like:

[code]
String defaultValue = xpath.evaluate("@default", colNode);
String classType = xpath.evaluate("@type", colNode);
if (classType.equals("xs:string")) {
col.setType(String.class);
if (!defaultValue.equals("")) {
col.setDefaultValue(defaultValue);
}
} else if (classType.equals("xs:decimal")) {
...
} else if (classType.equals("xs:integer") || classType.equals("xs:int")) {
...
} else if (classType.equals("xs:boolean")) {
...
} else if (classType.equals("xs:date") || classType.equals("xs:time") || classType.equals("xs.dateTime")) {
...
} else if (classType.equals("xs:unsignedByte")) {
...
} else if (classType.equals("jdnc:expression")) {
col.setType(Object.class);
String expression = xpath.evaluate("@expression", colNode);
col.setExpression(expression);
} else {
System.err.println("unexpected classType: '" + classType + "'");
}
[/code]

Huh, it looks like I forgot to add the expression field to the DataColumn. Also, the DataRow is that actual object that contains the values, so DataRow's getValue(DataColumn col) method would have to be changed so that if col is a calculated column (expression != null), then the expression will be evaluated and returned instead of the actual value.

The XML serialization/deserialization is a rough draft, not very pretty but works for the simple cases. I'm working on extending it to hold the relationships as well. Eventually I'll probably move the static methods from DataSet into DataSetUtils. That way some of the things Patrick has been working on (copying to the clipboard, CSV file formats, etc) can be used in the same way as the XML stuff.

Have fun!
Richard

dhall
Offline
Joined: 2006-02-17
Points: 0

Rich:

>> Minor nit: I'm just getting up to speed on xml schema myself, but I think you'd want the type to be something like "jdnc:expression" instead of "xs:expression" because the latter implies that there is an expression type in the xmlschema namespace. Maybe change the attribute name "default" to "expression" or something too. Not really important, we can work the specifics of that out later.

I'm sure you're absolutely correct -- what I should've said was that what XML I know is rusty and incomplete.

>> Huh, it looks like I forgot to add the expression field to the DataColumn. Also, the DataRow is that actual object that contains the values, so DataRow's getValue(DataColumn col) method would have to be changed so that if col is a calculated column (expression != null), then the expression will be evaluated and returned instead of the actual value.

I'll study the source some more, and probably have more questions in a day or two. Let me know if you update something that'll affect any of this, as I refresh my tree fairly infrequently.

Dave

dhall
Offline
Joined: 2006-02-17
Points: 0

Rich:

I just read in the documentation that java keywords are to be allowed as legal column names?? This is going to be a problem, in that what I'm already using is a java expression grammar: words like 'null', 'this', 'true', 'false', 'instanceof' will be ambiguous.

Also, any word that can be interpreted as a classname may be ambiguous: if they're not ambiguous, there'll still have to be a lot of hashing around to determine if, for example, Number refers to the java.lang.Number class or to a column called 'Number'. The jga parser imports all of the classnames in the java.lang package, so 'Object', 'String', 'Class', 'Number', 'System', and the like will also be problems.

Dave

Richard Bair

>I just read in the documentation that java keywords are to be allowed as
>legal column names?? This is going to be a problem, in that what I'm
>already using is a java expression grammar: words like 'null', 'this',
>'true', 'false', 'instanceof' will be ambiguous.
>
>Also, any word that can be interpreted as a classname may be ambiguous: if
>they're not ambiguous, there'll still have to be a lot of hashing around to
>determine if, for example, Number refers to the java.lang.Number class or
>to a column called 'Number'. The jga parser imports all of the classnames
>in the java.lang package, so 'Object', 'String', 'Class', 'Number',
>'System', and the like will also be problems.

Hmmm....Of course databases have a slew of words that are keywords, though
several allow us to get around that problem by wrapping the column name in
quotes, if necessary. So what if:

Number

meant java.util.Number and:

"Number"

meant to use the column name?

Richard

---------------------------------------------------------------------
To unsubscribe, e-mail: jdnc-unsubscribe@jdnc.dev.java.net
For additional commands, e-mail: jdnc-help@jdnc.dev.java.net

dhall
Offline
Joined: 2006-02-17
Points: 0

> Hmmm....Of course databases have a slew of words that
> are keywords, though
> several allow us to get around that problem by
> wrapping the column name in
> quotes, if necessary. So what if:
>
> Number
>
> meant java.util.Number and:
>
> "Number"
>
> meant to use the column name?

It's halfway there: we'd then have to figure out how to represent the string literal 'Number' -- databases frequently use single quotes for string literals (with a pair of single quotes standing for an embedded single quote), but since I'm working with a java grammar, that may not be easy to support.

I'll have to look into it, as it likely means changes to the lexical structure. I don't know how easy it'll be to override the lexer within an existing javacc implementation. If it was just the java keywords, it'd be easier to say 'don''t use those for column names', but a lot of the classnames are somewhat reasonable column names as well.

Say, aren't most database columns case-insensitive? column names in all caps would work ;)

Dave

Richard Bair

>Say, aren't most database columns case-insensitive? column names in all
>caps would work ;)

Oh, I just thought of something. In the DataProvider stuff we allow for
parameterized providers, like using params in an SQL query for instance. In
those we use ":paramName" in the traditional way. Could possibly use
":colName" for the columns. Somewhat different semantics from the provider
though...

Some others

${colName}
"colName" (with literals being 'literal' and it's being 'it''s')
*colName*
#colName
[someothe whacked out character]colName
tableName.colName

Hmm...with the last one you can get pretty close. Still, since BigDecimal in
java 5 has a ZERO property BigDecimal.ZERO might be legal in the script, as
is a Table named BigDecimal with a column named ZERO.

Richard

---------------------------------------------------------------------
To unsubscribe, e-mail: jdnc-unsubscribe@jdnc.dev.java.net
For additional commands, e-mail: jdnc-help@jdnc.dev.java.net

Patrick Wright

> ${colName}
> "colName" (with literals being 'literal' and it's being 'it''s')
> *colName*
> #colName
> [someothe whacked out character]colName
> tableName.colName
>
> Hmm...with the last one you can get pretty close. Still, since BigDecimal
> in
> java 5 has a ZERO property BigDecimal.ZERO might be legal in the script,
> as
> is a Table named BigDecimal with a column named ZERO.

We can't protect developers from themselves. The last two options would
work for me, personally. I think that this should be release-specific,
with some way for developers to workaround it. Release-specific means that
for a given JDNC release, we actually document what the magic naming
convention is, so if errors occur, they know where to look. Workaround
means--it would be good if developers could have a way to declare their
own naming convention if they have a table named BigDecimal with a column
named ZERO. I have no proposal, though...

I want to write up some notes about scripting/EL support, but here's one
that applies. Taking the simple example of BeanShell, when you evaluate a
script dynamically you pass in a context object which has references that
BSH can use in the script. So in jEdit, there is a buffer variable pushed
into the context, which refers to the current Buffer instance in jEdit.

My thought is that, for a calculated/dynamic column, the names of all
DataColumns in the DataTable are pushed in as variables, and can be
accessed by the script directly. Common functions are also "statically"
imported, so one can refer to Math.sin() as sin().

The DataColumns would be wrapped in a simple ImmutableDataColumn instance,
where setFoo() fails silently, so that the script for a dynamic column
cannot modify DataColumn values. Ditto for DataTable or DataSet references
given to a script (or which it can navigate to)--they are immutable.

So, for the "package" table in Adventure Builder, the script would get the
following variables in context
packageid
catid
location
price
name
description
imageuri

And thus a dynamic column that gives a discounted price could be simply
defined as
price - (price * 0.10)

There would be other rules, not sure how to implement them, but: no access
to java.io, java.net or java.nio in the script; no use of "=" and ??

The idea is that these scriptlets run in a "sandbox", which is essentially
defined by the table they are defined in.

Patrick

---------------------------------------------------------------------
To unsubscribe, e-mail: jdnc-unsubscribe@jdnc.dev.java.net
For additional commands, e-mail: jdnc-help@jdnc.dev.java.net

dhall
Offline
Joined: 2006-02-17
Points: 0

Patrick, Richard:

Quick step back for a second: I've been proposing a java-based grammar
primarily because it is what I happen to have lying around. It
supports what I think of as my target audience, and I think there's
something to be said for using a java grammar within a tool targeted
at new java programmers as well, albiet a slightly extended grammer in
both cases (what I've got, and what we're trying to design).

That said, I think we can keep in mind the possibility that we may
want to use a simpler grammar: not having to support java methods,
constructors, member variables would be another way to resolve the
ambiguity. That will, quite obviously, take longer to implement, but
having done one, I think it won't take as long to implement as it will
to come to agreement on the grammar. If we go that direction, I'm
still willing to do the implementation: it serves my purposes with jga
to have mulitple languages all compiling to functors.

Whichever way we go, this discussion has already yielded a couple of
things that I'm folding back into jga's parser. I've got a first cut
working that does the computed column, but it does not provide any of
the database related syntax: that'll require extending the parser.

> I want to write up some notes about scripting/EL support, but here's one
> that applies. Taking the simple example of BeanShell, when you evaluate a
> script dynamically you pass in a context object which has references that
> BSH can use in the script. So in jEdit, there is a buffer variable pushed
> into the context, which refers to the current Buffer instance in jEdit.

jga's parser does something similar: there's a method

[code]
parser.bindThis(Object binding)
[/code]

that ties a specific object into the parser for resolution of method calls
and member access. It allows the user to use expressions like:

[code]
this.getColumn("price");
[/code]

> My thought is that, for a calculated/dynamic column, the names of all
> DataColumns in the DataTable are pushed in as variables, and can be
> accessed by the script directly. Common functions are also "statically"
> imported, so one can refer to Math.sin() as sin().

Resolving the names really isn't much of an issue, so long as the names
are not ambiguous with an existing namespace. I left myself a hook in
the name resolution logic that allows for relatively simple name
processing -- the worksheet uses an extended parser that uses this hook
to add 'cell', 'row', and 'col' keywords.

As far as things like Math.sin(), however, I think it depends. If we
stick with a Java grammar, then I think we can leave the static
methods where they are, both because the audience won't find their use
to be terribly daunting and because it brings a lot more names under
one namespace, causing more potential collisions. If we go with a
different grammar, then its just a matter of putting together a list
of the functions that we want to appear as keywords.

> The DataColumns would be wrapped in a simple ImmutableDataColumn instance,
> where setFoo() fails silently, so that the script for a dynamic column
> cannot modify DataColumn values. Ditto for DataTable or DataSet references
> given to a script (or which it can navigate to)--they are immutable.

This is a good idea for the first release: we can come back to it
later to see if we want to allow for a batch updating tool with UI
hooks.

> So, for the "package" table in Adventure Builder, the script would get the
> following variables in context
> packageid
> catid
> location
> price
> name
> description
> imageuri

> And thus a dynamic column that gives a discounted price could be simply
> defined as
> price - (price * 0.10)

I think this is pretty much where we're headed, in effect, it should
look to the user like you describe here. We're just working out the
lexical rules associated with the price token.

> There would be other rules, not sure how to implement them, but: no access
> to java.io, java.net or java.nio in the script; no use of "=" and ??

One general comment: I don't know that what we're doing with any of
this rises to the level of a 'script'. The thing that I notice is
that all of the functor related things: filters, validators, binders,
constraints, computed values, relationships, initializers, etc.
are all typically implemented in a single line of strategically placed
code (in some cases, there is some additional information wrapped
around the code, but the behaviour boils down to a single expression).

That said, it wouldn't be hard to restrict access to classes in
some set of specific packages, but why would we want to do so?

> Could possibly use ":colName" for the columns. Somewhat different
> semantics from the provider though...

> Some others

> ${colName}
> "colName" (with literals being 'literal' and it's being 'it''s')
> *colName*
> #colName
> [someothe whacked out character]colName
> tableName.colName

I think we'll end up having to support the last one in the syntax for
the DataValue expressions: we could use it in both places, which would
mean that Patrick's example would read

package.price * 0.90

It doesn't affect any lexical structure: just interpretation of bare
names linked by periods. WRT all of these, I didn't get a chance last
night to look at how to override the lexical rules associated with
tokens in a javacc implementation, if such a thing is possible.

Dave Hall
http://jga.sf.net/
http://jroller.com/page/dhall/

Patrick Wright

Dave

What I've been feeling is that in the short term, we should just grab one
of the many scripting languages that run in the JVM and attach it where
necessary to JDNC.

If you add the features you're suggesting, and want to propose it, that
seems cool. It's nice, actually, that you're writing to the list with your
ideas for what you're personally working on. On the other hand, if we can
get BeanShell computing columns for us with say, 1-2 days work, why not do
that?

I don't mean this to diss you or your skills or your interest. Just trying
to be practical about it.

One idea is that, if your language can work under the Bean Scripting
Framework (Apache/Jakarta) then we can swap it in easily as well.

Patrick

---------------------------------------------------------------------
To unsubscribe, e-mail: jdnc-unsubscribe@jdnc.dev.java.net
For additional commands, e-mail: jdnc-help@jdnc.dev.java.net

dhall
Offline
Joined: 2006-02-17
Points: 0

Rich:

>> In the incubator under reorg/src/java/org/jdesktop/dataset you will find the current working code for the DataSet. A DataSet is a kind of container for DataTables, which can have DataRelations established between them -- mimicking the relational model.

>> A Column in a DataSet can be "synthetic", meaning that values in that column are calculated. So I could have a Price column, a Qty column, and a calculated Price * Qty column.

>> A DataSet can also have a DataValue. This object represents a single value, and that value is the result of some expression. So, maybe I have a DataValue who's expression is "sum(orderTable.extPriceColumn)".

I can't find that particular path. I did find a collection of classes at reorg/binding/src/org/jdesktop/dataset that looks like what you're describing. In there, I found a DataValue that includes this code:

[code]
public Object getValue() {
if (expression != null) {
//TODO Hook functor stuff in here for expressions
return null;
} else {
return null;
}
}
[/code]

Looks like from here we can get to the enclosing DataSet, either by binding the dataset to a functor argument or by reflecting on the DataValue object. One of the features of the FunctorParser in jga is that you can tell the parser what "this" is, essentially giving it a context from which to interpret method/member access -- we could bind the DataSet to 'this' and then parse strings that include calls through the DataSet API to whatever we can reach from there.

That would be enough for proof of concept -- the better solution is to resolve the table and column names: the resulting functor will still call DataSet.getTable() and DataTable.getValue(row, columnname) via the reflection functors, but that can be nicely hidden behind more direct syntax. (Note: this came up in one of the other thread -- column names can't have embedded spaces if we want to use them as language tokens). We can also have special functors specific to navigating the data model (Hacker's Worksheet uses a special functor to represent cell references, and it works out rather well).

Do you have a demo app that uses the data model? I'm not seeing anything in the incubator, but I could've just missed it. If there's some demo code I can hack away on, I might be able to get some kind of parser going reasonably soon. I'd especially like to see how this gets described in the meta-data, as this is the likely the place where the language strings get inserted.

Dave Hall
http://jga.sf.net/
http://jroller.com/page/dhall/

rbair
Offline
Joined: 2003-07-08
Points: 0

Morning Dave,

> I can't find that particular path. I did find a
> collection of classes at
> reorg/binding/src/org/jdesktop/dataset that looks
> like what you're describing. In there, I found a
> DataValue that includes this code:
>
> [code]
> public Object getValue() {
> if (expression != null) {
> //TODO Hook functor stuff in here for
> here for expressions
> return null;
> } else {
> return null;
> }
> }
> [/code]

Ya, that's it. Sorry about this misleading path.

> Looks like from here we can get to the enclosing
> DataSet, either by binding the dataset to a functor
> argument or by reflecting on the DataValue object.
> One of the features of the FunctorParser in jga is
> s that you can tell the parser what "this" is,
> essentially giving it a context from which to
> interpret method/member access -- we could bind the
> DataSet to 'this' and then parse strings that include
> calls through the DataSet API to whatever we can
> reach from there.

Good. The DataSet would be a good place to start from, alright. You can, of course, get to anything through it.

> That would be enough for proof of concept -- the
> better solution is to resolve the table and column
> names: the resulting functor will still call
> DataSet.getTable() and DataTable.getValue(row,
> columnname) via the reflection functors, but that can
> be nicely hidden behind more direct syntax. (Note:
> this came up in one of the other thread -- column
> names can't have embedded spaces if we want to use
> them as language tokens). We can also have special
> functors specific to navigating the data model
> (Hacker's Worksheet uses a special functor to
> represent cell references, and it works out rather
> well).

I actually read that thread last week and added code in the name-generation and name-setting methods to ensure that this is the case. You can be sure of valid names (no whitespace).

> Do you have a demo app that uses the data model? I'm
> not seeing anything in the incubator, but I could've
> just missed it. If there's some demo code I can hack
> away on, I might be able to get some kind of parser
> going reasonably soon. I'd especially like to see
> how this gets described in the meta-data, as this is
> the likely the place where the language strings get
> inserted.

Ya, nearby in the reorg there is a "demos" project. In it you will find a package (org.jdesktop.demos.ab) that contains the adventure builder demo. Since I've been just finishing up this major refactoring, I don't remember what state the demo is in (whether it builds or not). Let me know if you have any problems running it.

Cheers
Richard

dhall
Offline
Joined: 2006-02-17
Points: 0

>> I actually read that thread last week and added code in the name-generation and name-setting methods to ensure that this is the case. You can be sure of valid names (no whitespace).

Excellent -- that'll be a help.

>> Ya, nearby in the reorg there is a "demos" project. In it you will find a package (org.jdesktop.demos.ab) that contains the adventure builder demo. Since I've been just finishing up this major refactoring, I don't remember what state the demo is in (whether it builds or not). Let me know if you have any problems running it.

I saw those classes, but didn't realize that the demo code would be so small. I'm finding building the reorg tree an interesting experience, being without a netbeans environment and a "C:\" drive. I'll slog through it, although I may have to beat it into submission. It'll make a good blog entry, at least.

Dave

dhall
Offline
Joined: 2006-02-17
Points: 0

Just a quick update, and pointer incase anyone else is trying to build the reorg tree without netbeans. I wrote a user.properties file that contains a bunch of stuff that apparently the NetBeans environment defines:

[code]
reorg.home=/home/dave/projects/incubator/reorg
reorg.lib=${reorg.home}/lib

application.args=
javac.debug=true
javadoc.preview=true
project.swing=${reorg.home}/swing
javac.source=1.5
javac.target=1.5

platform.active=1.5
platforms.1.5.home=/opt/jdk1.5.0
platforms.1.5.bootclasspath=
platforms.1.5.compile=
platforms.1.5.java=/opt/jdk1.5.0

main.jars=${reorg.lib}/commons-beanutils.jar:${reorg.lib}/commons-collections.jar:\
${reorg.lib}/commons-logging.jar
demo.jars=${reorg.lib}/hsqldb.jar:${reorg.lib}/jta-api.jar:${reorg.lib}/rowset.jar
test.jars=${reorg.lib}/junit.jar
swing.jar=${project.swing}/dist/swing.jar

javac.classpath=${main.jars}:${demo.jars}:${test.jars}:${swing.jar}
[/code]

Then, I defined an environment variable

[code]
$ export ANT_ARGS=-Duser.properties.file=/home/dave/projects/incubator/reorg/user.properties
[/code]

Ant's got much better classpath handling than the current netbeans based build scripts use: essentially, to get the script to work, I had to type out the entire classpath, rather than define it as an ant path-like structure and reference it. This is because it is passed as an argument to the javac macrodef.

Is it just me, or if I'd waited a week, the main tree would include all the code and I'd only have to build the demo?

Please don't make the main tree use this build method -- it's not really ready for group development, and IMO, imposing NetBeans as a required development environment is a non-starter.

Dave Hall
http://jga.sf.net/
http://jroller.com/page/dhall/

dhall
Offline
Joined: 2006-02-17
Points: 0

>> Is it just me, or if I'd waited a week, the main tree would include all the code and I'd only have to build the demo?

As I suspected -- I should've just waited a week.

I've cloned the demo out of the newly released tree and started poking around.

It looks like the likely place to start will be in the adventure/resources/dataset.xsd file. I thought I'd add a

element to put a computed column in the table. I believe that it'll entail defining the expression type, but my XML is fairly rusty -- it appears that an expression type element needs to be defined in AdventureDS.xsd, but that isn't in the source tree anywhere that I can find.

If I'm offbase here, is there a better place to start? It looks from this code in MainWindow.java

[code]
TabularDataModelAdapter activitiesDetailModel = new TabularDataModelAdapter(ds.getTable("activitiesDetail"));
new DirectTableBinding(activitiesTBL, activitiesDetailModel, new String[]{"name", "description", "price" });
[/code]

like what I want to do is add "discount" to the column list and somehow get "price * 0.10" into the meta data.

Dave Hall
http://jga.sf.net/
http://jroller.com/page/dhall/

Patrick Wright

David

Related question on functors--if I want to test incorporating functors in
my code, what version of JGA do you recommend? There is a jar in the
incubator under lib/dhall I think, then there is CVS for your project
home, and the 0.6 release jar...

I want to create an import/export filter with your library, so I need to
use a version that is both stable and that you recommend to use if I write
a WebStart demo.

Thanks
Patrick

---------------------------------------------------------------------
To unsubscribe, e-mail: jdnc-unsubscribe@jdnc.dev.java.net
For additional commands, e-mail: jdnc-help@jdnc.dev.java.net

dhall
Offline
Joined: 2006-02-17
Points: 0

Patrick:

What's in the incubator is more current than the last 'Current Release', although I've had several subsequent 'previews'. The last preview release was a few weeks ago, and is available at the site, although none of the convenient links on the home page link to it.

If you follow this link

https://sourceforge.net/project/showfiles.php?group_id=49942

you can get the latest preview and its associated javadoc. This is reasonably close to the impending 0.7 release, which I'm in the late stages of preparing (although that's on hold temporarily while I ramp up my participation around here).

The most 'stable' alternative is the 0.6 -- nothing undeprecated in that release will be removed in 0.7 (although a few things may be deprecated and will be removed in 0.8). The only things that may change dramatically between the preview releases and 0.7 is new stuff added since 0.6.

If you're primarily interested in using the functor collection, the differences between 0.6 and 0.7 are 1) removal of old method names that had been deprecated in 0.6, 2) addition of a couple of dozen functors used to implement some of the lower level syntax of the functor parser, and 3) a few bug fixes (primarily related to error messages)

0.6 doesn't have the functor parser -- if you're interested in using it, grab the latest preview.

I'm looking forward to hearing how it works for you.

Dave Hall
http://jga.sf.net/
http://jroller.com/page/dhall/

dhall
Offline
Joined: 2006-02-17
Points: 0

Patrick

>> A question is how to do this efficiently. At the very least, we might have
>> a rule that expressions can neither have side-effects nor can they depend
>> on any global context...what I mean is that for a given expression, based
>> only on the values of the columns it depends on, for a given input of col
>> (x,y,z) with values (n,p,q) it always returns the same result. Can't
>> remember what that's called, silly me...anyway, if we have that, we have a
>> hope of caching these values...otherwise people will try creating a
>> "christmas tree" form with 10,000 cells, and attach expressions to all of
>> them, and even with the data not changing it will be horribly inefficient.
>>

When I put together the first couple of version of my spreadsheet, I hadn't
even considered the idea of side effects: analysing an expression to determine
if there are any side-effects isn't easy, particularly in the face of method
calls. Heuristically, we can assume that any method that starts with 'set'
may affect the value of the object on which it is called, as can any assignment
to a member, but determining that methods like java.awt.Point.move(x,y) change
the value of the point in question is pretty tough.

I'm fairly certain that the common usage will be pretty reasonable: there'll
always be someone that breaks a system by doing unreasonable things. In all
probability, so long as we've cached the results (and I really wouldn't
anticipate _not_ caching the results of every data change), performance should
stay pretty reasonable.

>> I think there are two features here. Given some range of rows and cells,
>> apply operation x; and, create virtual "groups" of rows (or cells) based
>> on some expression. I think the most common use case is to create one or
>> more "groups" of rows (forget about cols), grouping by values in 1..n
>> columns: e.g. group by year of birth, group by first letter of last name,
>> group by gender. So one problem, which is really just a data problem (e.g.
>> has nothing to do with calculated fields in a group, or headers/footers),
>> is how we represent virtual groups. I'm not sure what that would even mean
>> for trees (though I'm sure there's a way to look at it), so right now I'm
>> just thinking of tabular data sets.
>>

You're right: they're two different things, but I tend to think
they're somewhat related in that they both deal with sets of records.
The important efficiency question is how many passes through the set
do you end up doing: if we can keep it at one per major user decision
(change the search filter, pull up a new header record, etc) we'll be
OK. Multiple passes to compute membership in multiple groups or to
compute multiple summary fields will be a problem: any good algorithm
library (and I hope jga falls into that category) will have tools that
you can use to write that code, but to find a way to expose it in a
GUI builder will be interesting.

BTW: I did a TreeModel once based on just the situation you've described:
unique values in one column were one level of tree nodes, unique
values in a second column were a subordinate level of nodes (and of course
this could repeat as necessary) and the leaf nodes were the rows with the
given values. There were other tree model forms that my former coworkers and I
identified -- hierarchical relationships embodied in the data via parent/child
fields, this 'outline' form, a 'naked objects' form where the data is a list
of records, and the leaf nodes are the individual properties of the tree node
records, a fully-described menu form where every tree node had an individual
meaning and description.

>> It seems like a group is just a subset of 0..n rows from the main dataset.
>> Something (the grouping evaluator) has to give you those rows. The input
>> for the grouping evaluator is itself a list of rows, which means that
>> groups can nest inside groups. A group would (maybe) register itself as a
>> listener for the columns on which it depends, to automatically re-group if
>> any change.
>>
>> Your functor operations are then just operations on a list of rows, which
>> it sounds like you have already.
>>

I believe that I have the necessary functors and algorithms implemented. The
parser may need to grow a couple of things to enable support via an expression
language.

>> The display of groups really seems like it is not an issue (I know, I'm
>> being cavalier), in the sense that, virtual groups and header/footer
>> "virtual rows" are just part of an alternate way of looking at the model.
>> In fact, if you instantiate and load a tabular data model, you could then
>> strap on a set of group modellers which provide grouping/set organization,
>> and which offer the "virtual rows" of headers and footers. The question of
>> grouping is orthogonal to the data model itself. It's also an aspect of
>> the "data" (termed loosely) which may be ignored on some UIs and shown on
>> other UIs; in other words, if I have a tabular data set, I shouldn't be
>> forced to display it with groups if groups were attached. The underlying
>> data set is really unaffected by grouping. Grouping is an alternate view
>> of the underlying data which is available for UIs to construct themselves
>> with virtual rows/headers/footers/computed columns.
>>

I agree with all that: it seems to me to play more into how much the filtering
layer (whether its the current filter pipeline or something else) can be
exposed to user manipulation than into the bindings. It seems fairly natural
to be able to apply summary functions to the currently displayed set of data,
but I think that manipulating the parameters that determine membership in the
currently displayed set can be a different mechanism (or a similar mechanism
applied in a different place in the design tool).

Dave Hall
http://jga.sf.net/
http://jroller.com/page/dhall/

PS: I really didn't intend to sit down and right a small book tonight, it just
sorta happened. I can't believe this didn't break the page!

Patrick Wright

Dave


> When I put together the first couple of version of my spreadsheet, I
> hadn't
> even considered the idea of side effects: analysing an expression to
> determine
> if there are any side-effects isn't easy, particularly in the face of
> method
> calls. Heuristically, we can assume that any method that starts with
> 'set'
> may affect the value of the object on which it is called, as can any
> assignment
> to a member, but determining that methods like java.awt.Point.move(x,y)
> change
> the value of the point in question is pretty tough.

Yes, I know it is difficult. I do recall that when I worked with tabular
data sets a lot, calculated columns (on the client) were a must-have. And,
more importantly, in the tools I used, didn't usually impact performance
noticeably--grouping could, and did sometimes, but normal calculated
columns did not.

One point is that the caching has to be intelligent enough that the
determination to recalculate happens on the individual column level, where
the expression is bound. If col B = round(A, 0) * 1.5, and col c = round(B
/ 2), the question is whether C is re-evaluated every time A is changed,
even if B has the same value (say A is changed from 1.1 to 1.2). In a
spreadsheet, we do this sort of chaining all the time.

My guess is that all of this has to be pluggable into a DataSet, so that
different evaluation/calculation engines can be tried out, perhaps even
different ones for different cases. Richard's idea of a DataValue fits in
here, but what I'm thinking is that calculated DataValues get passed to an
"engine" that is listening to model changes and running these through the
calculations in the most efficient way possible. One binds the "engine" to
the DataSet, and DS calls out to it when necessary to pick up column
values. That way the engine can be either efficient, or easy to use, or
optimized for speed and not memory, etc. There is some inefficiency in
that (as the engine needs to call back to the DS to pick out underlying
values), but it takes it out of the critical path for DS development.

Again we have to identify use cases for this stuff. On a single row
operation, in a single-row form (backed by a tab. DS, but displaying a
single row) and where the calculated column does not include any range
expressions (e.g. operates only on the current row), my guess is there
will never be any problem in reasonable use for performance. The worrying
thing is really tables, and in particular, tables where the underlying
data can be changed.

>
> I'm fairly certain that the common usage will be pretty reasonable:
> there'll
> always be someone that breaks a system by doing unreasonable things. In
> all
> probability, so long as we've cached the results (and I really wouldn't
> anticipate _not_ caching the results of every data change), performance
> should
> stay pretty reasonable.

Well, use cases, over and again. On the mailing list we should have a
number of people who have uses for calculated columns/expressions and we
should poll them to see what they are used for. I once worked on a project
where this young economist built an actuarial model based on a Monte Carlo
simulation using Excel and VBA. My guess is that won't be one of our use
cases :), though it is interesting to ask why not...

> You're right: they're two different things, but I tend to think
> they're somewhat related in that they both deal with sets of records.
> The important efficiency question is how many passes through the set
> do you end up doing: if we can keep it at one per major user decision
> (change the search filter, pull up a new header record, etc) we'll be
> OK. Multiple passes to compute membership in multiple groups or to
> compute multiple summary fields will be a problem: any good algorithm
> library (and I hope jga falls into that category) will have tools that
> you can use to write that code, but to find a way to expose it in a
> GUI builder will be interesting.

All the RAD/report tools I'm thinking of supported grouping. The group
definition was pretty simple: the group is based on defining a group key,
which is a set of columns in a particular order. The key groups rows into
a set. So: "Insert/New Group", set columns, OK. You get a group header and
footer for editing and putting in group-related columns. Nested groups
work the same way.

What is interesting is that the range functions can work on groups (e.g.
sum(salary) for group 1), so groups need identifiers.

Patrick

---------------------------------------------------------------------
To unsubscribe, e-mail: jdnc-unsubscribe@jdnc.dev.java.net
For additional commands, e-mail: jdnc-help@jdnc.dev.java.net

rbair
Offline
Joined: 2003-07-08
Points: 0

Wow, piles of stuff to talk about. I'll just focus on part of it though (time, time, time, where have you gone?).

In the incubator under reorg/src/java/org/jdesktop/dataset you will find the current working code for the DataSet. A DataSet is a kind of container for DataTables, which can have DataRelations established between them -- mimicking the relational model.

A Column in a DataSet can be "synthetic", meaning that values in that column are calculated. So I could have a Price column, a Qty column, and a calculated Price * Qty column.

A DataSet can also have a DataValue. This object represents a single value, and that value is the result of some expression. So, maybe I have a DataValue who's expression is "sum(orderTable.extPriceColumn)".

With this in mind...

> 1) filters used to dynamically exclude rows from the
> complete set, based
> on values in any of the columns of that row (e.g.
> price > 1000 and
> inventory > 0).

Right, reason #1

> 2) There are expressions used as calculated column
> values, which display
> as any other column but are not editable, and are
> dynamically update as
> the dependent data is changed elsewhere in the table.
> The fact they are
> dynamic means you can't just pull them in a server
> request.
>
> 3) Calculated column values used for sorting.
>

As specified above, it might be possible to express this on the JXTable level, but it might also be possible to express this on the DataSet level. I'm not sure which is better, or will be easier. I think this kind of computation needs to be supportable on the JXTable level, simply because otherwise a GUI designer would have to constantly go back to the DataSet provider guy and have him add DataValues and calculated DataColumns. OTOH, since the DataSet is so close to the data it is really easy to make these kinds of computations on the DataSet level.

> 4) Expressions used to control simple aspects of a
> column/row's display:
> bold face, color, maybe font.

Hmmm... the benefit would be, no custom renderers? What about something like:

[code]
dataValue.setExpression("\"\" + custTable.lastName + \"<\b>, \" + custTable.firstName");
[/code]

This would simply construct an HTML string which would be fed to the JLabel and viola. I'm sure the syntax isn't quite right, but the idea I think works well.

> 5) Group header/footer rows for summary values; can
> use calculated
> expressions here, but can use range functions
> (SUM(price)) across the set.

Not necessary if the DataValue in the DataSet is used.

Summary:

There are 3 initial places where some kind of simple expression language could be really useful in JDNC. I hasten to add that their utility is really oriented around a GUI Builder experience for developers. A good GUI builder experience has a limited amount of code that the developer has to write.

1) for defining calculated columns and DataValues in the DataSet

2) for defining renderers for Lists & Tables

3) for defining a calculated result for a read-only component like a JLabel

Of course all of these things could be defined in code, but it would be really nice to have a single "expression" field in a Property Sheet that somebody could type "price * qty" into, instead of having to write some interface and hook it in with the DataValue, etc.

The other stuff would be fun to explore as well.

Richard

dhall
Offline
Joined: 2006-02-17
Points: 0

Richard:

>> A Column in a DataSet can be "synthetic", meaning that values in that
>> column are calculated. So I could have a Price column, a Qty column,
>> and a calculated Price * Qty column.
>>
>> A DataSet can also have a DataValue. This object represents a single
>> value, and that value is the result of some expression. So, maybe I
>> have a DataValue who's expression is "sum(orderTable.extPriceColumn)".
>>

I'm always leary of 1:1 relationships, so the idea that there's one
and only one summary value associated with a DataSet raises a flag.
Of course, that's based solely on your no-doubt very abbreviated
description here.

>> > 2) There are expressions used as calculated column
>> > values, which display
>> > as any other column but are not editable, and are
>> > dynamically update as
>> > the dependent data is changed elsewhere in the table.
>> > The fact they are
>> > dynamic means you can't just pull them in a server
>> > request.
>> >
>> > 3) Calculated column values used for sorting.
>> >
>>
>> As specified above, it might be possible to express this on the
>> JXTable level, but it might also be possible to express this on the
>> DataSet level. I'm not sure which is better, or will be easier. I
>> think this kind of computation needs to be supportable on the JXTable
>> level, simply because otherwise a GUI designer would have to
>> constantly go back to the DataSet provider guy and have him add
>> DataValues and calculated DataColumns. OTOH, since the DataSet is so
>> close to the data it is really easy to make these kinds of
>> computations on the DataSet level.
>>

I don't anticipate that it'll be easier or harder in either place: to
make functors work, we need to work out what we're going to pass as
arguments and what you're going to bind ahead of time: in either case,
I think we'd start out with the assumption that we're going to get the
equivalent of a row of data somehow. The filters that I put into the
incubator work this way, I can't think of a compelling reason for computed
fields to work differently.

My gut feeling is that this is more of a model concern than a view
concern, so it falls more naturally into the data side of things than
the widget side of things.

OTOH, I'm not sure that we really need to make this a hard and fast
decision: I suspect that it'll be far more common that the same person
is wearing both hats, and in those cases I think we'd want to have a
preference that we teach, but that we'd allow the designer to put it
where it makes the most sense on a case by case basis. Note that if
you describe bindings by column names, and also allow column names in
your expression language, then the bindings are, in effect, constant
expressions (ie, all of the columns are, in effect, computed -- some
of them might also include arithmetic, logical, formatting, or string
concatenation expressions) (constant might be a bad term here:
unprocessed might be better, but it feels more awkward)

In cases where there is two people involved in the discussion, I
suspect that the dataset provider will want to provide computed data
using whatever tools are provided by the persistence mechanism before
configuring computations into the data set metadata, which puts more
weight behind this falling into the widget side.

Also, do we still have a JX layer that acts as extended swing classes
and a JN layer that is the base for the more advanced markup based
stuff? If so, I wonder if this belongs more on the JN layer than JX.

>> > 4) Expressions used to control simple aspects of a
>> > column/row's display:
>> > bold face, color, maybe font.
>>
>> Hmmm... the benefit would be, no custom renderers? What about something like:
>>
>> dataValue.setExpression("\"\" + custTable.lastName + \"<\b>, \" + custTable.firstName");
>>
>> This would simply construct an HTML string which would be fed to the
>> JLabel and viola. I'm sure the syntax isn't quite right, but the idea
>> I think works well.
>>
>>

That would work, but isn't quite the same thing. The benefit
is no custom renderers, but in my mind, it speaks to the collection
of highlighters: there could be a single highlighter class with different
expressions applied. I think it'll be easier on the markup if simple
expressions can be included inline, and a single highlighter class used
(whether its a single instance or not is another matter).

Severity == "CRITICAL" ? Color.RED : Color.BLACK
Priority == "EXTREME" ? new java.awt.Color(128, 64,64) : Color.WHITE

for as many extension points as you want to define: foreground, background,
font, icon, text, etc...

The markup processor can pass the string to a functor parser and pass the
resulting functor to a generic highlighter's constructor. The issue becomes
what value to pass to the highlighter when it is in use: the value in the
cell, the value in the row, or the index of the row (and presume that the
highlighter can get the correct value from the model).

>> > 5) Group header/footer rows for summary values; can
>> > use calculated
>> > expressions here, but can use range functions
>> > (SUM(price)) across the set.
>>
>> Not necessary if the DataValue in the DataSet is used.
>>

See my first comment: the 1:1 limitation (if it exists) is a little
restrictive here.

I'm not sure that we necessarily need to restrict the summary
computations to a special row in the table, although that'll be a
common request.

>> Summary:
>>
>> There are 3 initial places where some kind of simple expression
>> language could be really useful in JDNC. I hasten to add that their
>> utility is really oriented around a GUI Builder experience for
>> developers. A good GUI builder experience has a limited amount of code
>> that the developer has to write.
>>
>> 1) for defining calculated columns and DataValues in the DataSet
>>
>> 2) for defining renderers for Lists & Tables
>>
>> 3) for defining a calculated result for a read-only component like a JLabel
>>
>> Of course all of these things could be defined in code, but it would
>> be really nice to have a single "expression" field in a Property Sheet
>> that somebody could type "price * qty" into, instead of having to
>> write some interface and hook it in with the DataValue, etc.
>>

There are a few other places where user-configurable expressions may come into play:

- linkage between logical tables in the metadata

- default values when starting a new record to be added (especially
the hidden ID fields that tend to tie different records together)

Also: I think one of the nice features to using functors in the implementation
is that they can easily be live in your GUI tool with the same effects
that they will have in the runtime environment.

Dave Hall
http://jga.sf.net/
http://jroller.com/page/dhall

dhall
Offline
Joined: 2006-02-17
Points: 0

Here's a list of things that were suggested to me that I mentioned earlier

http://www.janino.net
http://lts.online.fr/dev/java/math.evaluator/
http://www.ognl.org/
http://www.hunter-lovell.org/freegraph/

Martyn Cutcher has a white paper on some efforts he's made to find common abstractions between xpath and sql at http://www.cutthecrap.biz/software/whitepapers/gpath.html.

dhall
Offline
Joined: 2006-02-17
Points: 0

Richard:

I've made a little progress, but it looks like I'm bumping into a problem on your end?

the DataTable.getValue(DataRow,DataColumn) method is returning strings from the price column, even though it looks like they should be returning BigDecimals. Is there something in the metadata that we can change?

I've got the expression down to

price * (BigDecimal) 0.15

for a column named 'tax'

Dave

Richard Bair

>the DataTable.getValue(DataRow,DataColumn) method is returning strings from
>the price column, even though it looks like they should be returning
>BigDecimals. Is there something in the metadata that we can change?

Yes indeed. In the schema the type for the price field is "xs:string". It
should be changed to "xs:decimal". The parser should then recognize it as a
BigDecimal.

>I've got the expression down to
>
>price * (BigDecimal) 0.15

Sweet. Can't wait to see it in action!

Richard

---------------------------------------------------------------------
To unsubscribe, e-mail: jdnc-unsubscribe@jdnc.dev.java.net
For additional commands, e-mail: jdnc-help@jdnc.dev.java.net

dhall
Offline
Joined: 2006-02-17
Points: 0

Hi Richard:

Naturally, I'd suggest that jga is everything you want, but I am quite biased. It's especially hard for me to describe without sounding like a product pitch, even though I'm not a marketer.

I don't know of another implementation of a language parsing down to compound functors per se. I believe that idea to be somewhat original -- although influenced by similar looking things in many other languages, of course: I don't know of any that retain Java's strong typing and produce objects that are then independant of the parser and generally reusable within normal looking classes.

You might want to look at lambda4j, at http://savannah.nongnu.org/projects/lambda4j/
although it is based on a preprocessor. I don't know if any of the 'dynamic scripting' languages that are getting so much attention can/will work, although I wouldn't be suprised if a couple of them can be made to work. I suspect, however, that they are all aiming higher than this: many of them want to produce new classes based on their scripts, and I believe that what's appropriate in this case is something quite a bit more finely grained.
I'll try and pull together a couple of other pointers tonight: several people pointed me to various parser/expression evaluator projects after having read in my README file that I intended to look in that direction -- some of those pointers might be useful to you.

As to what jga has, I wrote a FunctorParser that produces compound functors based on a Java expression grammar. The goal of the parser was that anything you can assign to a variable in Java you should be able to parse to a Functor. I believe that makes the language easy to learn. There's an early version in my incubator area, and I described it in this thread:

http://www.javadesktop.org/forums/thread.jspa?forumID=53&threadID=6560&m...

The syntax isn't hard to extend: what I included in the filter proposal is an extension of the base parser that adds a single keyword. The Java Hacker's Worksheet is based on another extension of the base parser that adds three spreadsheet-related keywords. I think it'd be very useful for the parser to know about the metadata, so that we could use names from the metadata in the way you're suggesting. I believe that I've left a couple of methods exposed to extensions that will allow that type of user-level naming resolution to be implemented, and if I didn't, that it isn't hard to add.

BTW: the status of JGA right now is that I'm polishing up the tree for a 0.7 release. However, the parser that will be released in 0.7 is pretty much already done, and if you go to the site and pull down the 0.6.5-preview, that version of the parser is probably (minus potential bug fixes that I don't know about at this point) what'll be included in 0.7. The version in the incubator is a couple of preview releases older and not quite as complete, as far as the java expression grammar goes.

Dave Hall
http://jga.sf.net/
http://jroller.com/page/dhall/

Patrick Wright

Dave

Jumping in here (and I know, ideas are cheap, if not free): I've been
hoping to squirrel away time to try to integrate some sort of dynamic
scripting with JDNC filters/sorts/display characteristics, but, you know.
I haven't. But here are a couple of things I've seen available in the
past.

1) filters used to dynamically exclude rows from the complete set, based
on values in any of the columns of that row (e.g. price > 1000 and
inventory > 0).

2) There are expressions used as calculated column values, which display
as any other column but are not editable, and are dynamically update as
the dependent data is changed elsewhere in the table. The fact they are
dynamic means you can't just pull them in a server request.

3) Calculated column values used for sorting.

4) Expressions used to control simple aspects of a column/row's display:
bold face, color, maybe font.

5) Group header/footer rows for summary values; can use calculated
expressions here, but can use range functions (SUM(price)) across the set.

6) Extension of group functions--range functions with embedded
expressions, like the IFF or IIF function in Excel.

General problems I've found:
- It's hard to find an expression language that works for the hard cases.
The toolkits I've used all let you do the above, but when you wrote
complex expressions it was easy to break, and there was no debugging
available.

- Need to be clear on how null values, empty strings, string comparison,
numeric/type promotion and datatype casts are handled.

- Easy to break if column names get out of sync.

- Hard to predict what the performance impact would be of using these as
you wrote them.

- In all cases, generally hard to write complex combinations of
groups/ranges/expressions.

Anyway, these are the aspects I hope to work on if/when I can make time. I
think the idea of using JGA might be better than one of the scripting
languages especially if JGA can offer more restricted options to the
developer; not a good idea to encourage a script to run every time
getValueAt() is called :).

Patrick

> Hi Richard:
>
> Naturally, I'd suggest that jga is everything you want, but I am quite
> biased. It's especially hard for me to describe without sounding like a
> product pitch, even though I'm not a marketer.
>
> I don't know of another implementation of a language parsing down to
> compound functors per se. I believe that idea to be somewhat original --
> although influenced by similar looking things in many other languages, of
> course: I don't know of any that retain Java's strong typing and produce
> objects that are then independant of the parser and generally reusable
> within normal looking classes.
>
> You might want to look at lambda4j, at
> http://savannah.nongnu.org/projects/lambda4j/
> although it is based on a preprocessor. I don't know if any of the
> 'dynamic scripting' languages that are getting so much attention can/will
> work, although I wouldn't be suprised if a couple of them can be made to
> work. I suspect, however, that they are all aiming higher than this: many
> of them want to produce new classes based on their scripts, and I believe
> that what's appropriate in this case is something quite a bit more finely
> grained.
> I'll try and pull together a couple of other pointers tonight: several
> people pointed me to various parser/expression evaluator projects after
> having read in my README file that I intended to look in that direction --
> some of those pointers might be useful to you.
>
> As to what jga has, I wrote a FunctorParser that produces compound
> functors based on a Java expression grammar. The goal of the parser was
> that anything you can assign to a variable in Java you should be able to
> parse to a Functor. I believe that makes the language easy to learn.
> There's an early version in my incubator area, and I described it in this
> thread:
>
> http://www.javadesktop.org/forums/thread.jspa?forumID=53&threadID=6560&m...
>
> The syntax isn't hard to extend: what I included in the filter proposal is
> an extension of the base parser that adds a single keyword. The Java
> Hacker's Worksheet is based on another extension of the base parser that
> adds three spreadsheet-related keywords. I think it'd be very useful for
> the parser to know about the metadata, so that we could use names from the
> metadata in the way you're suggesting. I believe that I've left a couple
> of methods exposed to extensions that will allow that type of user-level
> naming resolution to be implemented, and if I didn't, that it isn't hard
> to add.
>
> BTW: the status of JGA right now is that I'm polishing up the tree for a
> 0.7 release. However, the parser that will be released in 0.7 is pretty
> much already done, and if you go to the site and pull down the
> 0.6.5-preview, that version of the parser is probably (minus potential bug
> fixes that I don't know about at this point) what'll be included in 0.7.
> The version in the incubator is a couple of preview releases older and not
> quite as complete, as far as the java expression grammar goes.
>
> Dave Hall
> http://jga.sf.net/
> http://jroller.com/page/dhall/
> ---
> [Message sent by forum member 'dhall' (David Hall)]
>
> http://www.javadesktop.org/forums/thread.jspa?messageID=57462
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jdnc-unsubscribe@jdnc.dev.java.net
> For additional commands, e-mail: jdnc-help@jdnc.dev.java.net
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: jdnc-unsubscribe@jdnc.dev.java.net
For additional commands, e-mail: jdnc-help@jdnc.dev.java.net

dhall
Offline
Joined: 2006-02-17
Points: 0

Patrick:

That's an interesting list of useful stuff.

>> 1) filters used to dynamically exclude rows from the complete set, based on values in any of the columns of that row (e.g. price > 1000 and inventory > 0).

This is already in the incubator

>> 2) There are expressions used as calculated column values, which display as any other column but are not editable, and are dynamically update as the dependent data is changed elsewhere in the table. The fact they are dynamic means you can't just pull them in a server request.

>> 3) Calculated column values used for sorting.

Both of these are reasonably doable: between the hacker's worksheet and the generic table model (both included in the jga jar), I've covered computed values pretty well. I don't have any examples of sorting based on the computed value, but if we can get it into the data model, we can sort on it. Right now, the threading in the worksheet and the caching in the generic table model are both naive, but its fixable without too much trouble.

>> 4) Expressions used to control simple aspects of a column/row's display: bold face, color, maybe font.

I've been thinking about this too: it's certainly possible to evaluate an expression that yields a font, fontstyle, or color, and to allow a renderer to use several such expressions as extension points. With the worksheet, I'm expecting to allow the date/number/string format to be computed by an expression, and the XML form would be something like


new java.util.Date()

/* a line of java code that formats a date */

I'd left bringing up this case for later: it seemed like a harder sell.

>>5) Group header/footer rows for summary values; can use calculated expressions here, but can use range functions (SUM(price)) across the set.

>> 6) Extension of group functions--range functions with embedded expressions, like the IFF or IIF function in Excel.

Both of these, as you no doubt realize, are the problems. To do header/footers for an entire collection across multiple levels, or with batch-report style processing generally involves more than one expression. To do it once for the displayed list of stuff on the UI is possible and the performance shouldn't be a problem (depends on how exotic the function is), and if I can't already support it with stuff in JGA (not necessarily the parser), I don't believe that I'm too far off.

The parser already has the capability to import classes so that you don't have to fully qualify them. I think that it wouldn't be too hard to duplicate the static import of individual methods. JGA has several facades that expose the standard algorithmic implementations behind static methods, so static importation of the facades would have the effect of adding those algorithms to the language. If the UI has a filtered list, then we can pass the filtered list through one of the facades and perform a variety of computations: min, max, sum, and count are already supported directly, and average might be done efficiently (it can be done inefficiently -- by doing count and sum in two passes). The basic language already has conditional processing (the ?: operator), and I'm strongly leaning toward implementing the comma operator to allow for mulitple expressions in a single functor, both of which open up a light amount of processing possibilities.

I've put off adding range operations to the worksheet for now: I expect to get to it in 0.8. I expect that the worksheet will present range arguments to functions that expect them as a Collection of cell references.

Dave Hall
http://jga.sf.net/
http://jroller.com/page/dhall/

Patrick Wright

Dave

Thanks for the reply.


>>> 4) Expressions used to control simple aspects of a column/row's
>>> display: bold face, color, maybe font.
>
> I've been thinking about this too: it's certainly possible to evaluate an
> expression that yields a font, fontstyle, or color, and to allow a
> renderer to use several such expressions as extension points. With the
> worksheet, I'm expecting to allow the date/number/string format to be
> computed by an expression, and the XML form would be something like
>
>
> new java.util.Date()
>
/* a line of java code that formats a date */ >

>
> I'd left bringing up this case for later: it seemed like a harder sell.

A question is how to do this efficiently. At the very least, we might have
a rule that expressions can neither have side-effects nor can they depend
on any global context...what I mean is that for a given expression, based
only on the values of the columns it depends on, for a given input of col
(x,y,z) with values (n,p,q) it always returns the same result. Can't
remember what that's called, silly me...anyway, if we have that, we have a
hope of caching these values...otherwise people will try creating a
"christmas tree" form with 10,000 cells, and attach expressions to all of
them, and even with the data not changing it will be horribly inefficient.

> Both of these, as you no doubt realize, are the problems. To do
> header/footers for an entire collection across multiple levels, or with
> batch-report style processing generally involves more than one expression.
> To do it once for the displayed list of stuff on the UI is possible and
> the performance shouldn't be a problem (depends on how exotic the function
> is), and if I can't already support it with stuff in JGA (not necessarily
> the parser), I don't believe that I'm too far off.

I think there are two features here. Given some range of rows and cells,
apply operation x; and, create virtual "groups" of rows (or cells) based
on some expression. I think the most common use case is to create one or
more "groups" of rows (forget about cols), grouping by values in 1..n
columns: e.g. group by year of birth, group by first letter of last name,
group by gender. So one problem, which is really just a data problem (e.g.
has nothing to do with calculated fields in a group, or headers/footers),
is how we represent virtual groups. I'm not sure what that would even mean
for trees (though I'm sure there's a way to look at it), so right now I'm
just thinking of tabular data sets.

It seems like a group is just a subset of 0..n rows from the main dataset.
Something (the grouping evaluator) has to give you those rows. The input
for the grouping evaluator is itself a list of rows, which means that
groups can nest inside groups. A group would (maybe) register itself as a
listener for the columns on which it depends, to automatically re-group if
any change.

Your functor operations are then just operations on a list of rows, which
it sounds like you have already.

The display of groups really seems like it is not an issue (I know, I'm
being cavalier), in the sense that, virtual groups and header/footer
"virtual rows" are just part of an alternate way of looking at the model.
In fact, if you instantiate and load a tabular data model, you could then
strap on a set of group modellers which provide grouping/set organization,
and which offer the "virtual rows" of headers and footers. The question of
grouping is orthogonal to the data model itself. It's also an aspect of
the "data" (termed loosely) which may be ignored on some UIs and shown on
other UIs; in other words, if I have a tabular data set, I shouldn't be
forced to display it with groups if groups were attached. The underlying
data set is really unaffected by grouping. Grouping is an alternate view
of the underlying data which is available for UIs to construct themselves
with virtual rows/headers/footers/computed columns.

Is this making any sense? Haven't yet finished my first expresso.

>
> The parser already has the capability to import classes so that you don't
> have to fully qualify them. I think that it wouldn't be too hard to
> duplicate the static import of individual methods. JGA has several
> facades that expose the standard algorithmic implementations behind static
> methods, so static importation of the facades would have the effect of
> adding those algorithms to the language. If the UI has a filtered list,
> then we can pass the filtered list through one of the facades and perform
> a variety of computations: min, max, sum, and count are already supported
> directly, and average might be done efficiently (it can be done
> inefficiently -- by doing count and sum in two passes). The basic
> language already has conditional processing (the ?: operator), and I'm
> strongly leaning toward implementing the comma operator to allow for
> mulitple expressions in a single functor, both of which open up a light
> amount of processing possibilities.
>
> I've put off adding range operations to the worksheet for now: I expect to
> get to it in 0.8. I expect that the worksheet will present range
> arguments to functions that expect them as a Collection of cell
> references.

Cool.

Patrick

---------------------------------------------------------------------
To unsubscribe, e-mail: jdnc-unsubscribe@jdnc.dev.java.net
For additional commands, e-mail: jdnc-help@jdnc.dev.java.net

Patrick Wright

Another thought: a shortcut interface that might be useful would define a
set of standard display formatting (font/color/background) properties:
isBold(), getBackgroundColor(), getColor(), getFont(). On a given row one
would (somehow) bind the interface to a column (or, alternately, the the
row itself), using functors to return the property's value. In other
words, say for a table, instead of changing the display in at the level of
the component used for rendering the table cell, the component has
font/colors applied after being picked out by applying the properties to
it as read through the interface.

This would allow two things: the interface impl. can cache the objects
used for font/color, and second, the impl can be a listener for changes to
specific columns, and only re-apply the functor when a bound column
changes. Also, one might then be able to "attach" instances of this
interface to a form or table using a form-builder like NetBeans: which
means you just click the "isBold()" property and define the corresponding
functor for that column or a row (row being matched by some row-key).

I need to look through the classes and sketch out what this might look
like, in pseudo-code.

Hope this all doesn't sound (really) stupid :).

Patrick

---------------------------------------------------------------------
To unsubscribe, e-mail: jdnc-unsubscribe@jdnc.dev.java.net
For additional commands, e-mail: jdnc-help@jdnc.dev.java.net

Joshua Marinacci

I've been watching this dicussion but haven't added much not because it
isn't interesting but because I don't have the time. (Lots of cool
things coming up soon, so stay tuned.) I'd like to jump in a say a few
quick things though. First, if you want this type of style behavior you
really are talking about having an actual style system. I'd take a look
at the jeppers project ( https://jeppers.dev.java.net/ ) which has a
series of models and interfaces for representing the data, the grid
structure (size and spanning), the cell dimensions, the selections, and
the styles. There is are style objects which can be shared by multiple
cells, so you can call setBold(true) on the style that 4 cells share and
they will all be rendered appropriately. (Some of these style ideas may
be familiar to you Patrick. :)

I've started playing with Jeppers as the core of a spreadsheet
application (which I'll tell you all more about later), and it's pretty
well thought out. It also has a query language with references that
you'd want to take a look at. There are still some rough edges but I was
able to get a working spreadsheet up in a few hours with cell references
and basic calculations. (I tried to write my own expression parser too.
It took me 8 hours just to get arithmetic working so I decided to reuse
the hard work of others. Ahh, the beauty of open source. ;)

In all of the talk of functors I would suggest you limit your scope. If
the functors for tables get too advanced you will eventually either
reach a limit where it can't do any more, or else you'll reinvent SQL.
Figure out what actual developers will want to use the system for and
then design your functor/functional-language system to support that. You
may find that reusing an existing language better suits your needs.

Okay, back to your regularly scheduled posting.

oh, as an aside. I believe the function which always returns the same
value, that you spoke about Patrick, is the mathematical definition of a
function (as opposed to a 'method', which is what most OO languages
really have). I know that XPath was specificially designed to take
advantages of functions that are guaranteed to always return the same
value with the same inputs. That's how XSLTs can be precompiled to run
so quickly. That's also why there aren't any real variables.
(reinventing LISP, i tell ya! :)

back to the book
cheers,
- J

Patrick Wright wrote:
> Another thought: a shortcut interface that might be useful would define a
> set of standard display formatting (font/color/background) properties:
> isBold(), getBackgroundColor(), getColor(), getFont(). On a given row one
> would (somehow) bind the interface to a column (or, alternately, the the
> row itself), using functors to return the property's value. In other
> words, say for a table, instead of changing the display in at the level of
> the component used for rendering the table cell, the component has
> font/colors applied after being picked out by applying the properties to
> it as read through the interface.
>
> This would allow two things: the interface impl. can cache the objects
> used for font/color, and second, the impl can be a listener for changes to
> specific columns, and only re-apply the functor when a bound column
> changes. Also, one might then be able to "attach" instances of this
> interface to a form or table using a form-builder like NetBeans: which
> means you just click the "isBold()" property and define the corresponding
> functor for that column or a row (row being matched by some row-key).
>
> I need to look through the classes and sketch out what this might look
> like, in pseudo-code.
>
> Hope this all doesn't sound (really) stupid :).
>
> Patrick
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: jdnc-unsubscribe@jdnc.dev.java.net
> For additional commands, e-mail: jdnc-help@jdnc.dev.java.net

---------------------------------------------------------------------
To unsubscribe, e-mail: jdnc-unsubscribe@jdnc.dev.java.net
For additional commands, e-mail: jdnc-help@jdnc.dev.java.net

dhall
Offline
Joined: 2006-02-17
Points: 0

Joshua:

>> ... First, if you want this type of style behavior you really are
>> talking about having an actual style system. I'd take a look at the
>> jeppers project ( https://jeppers.dev.java.net/ ) which has a series
>> of models and interfaces for representing the data, the grid structure
>> (size and spanning), the cell dimensions, the selections, and the
>> styles. There is are style objects which can be shared by multiple
>> cells, so you can call setBold(true) on the style that 4 cells share
>> and they will all be rendered appropriately.
>>

I've looked at jeppers (its not the first time you've linked to it in
a discussion where I've mentioned my worksheet). It is a nicely done
control, and I'm thinking about porting my worksheet engine to the
jeppers grid instead of JTable, as an exercise in ensuring that my
functor-based engine can be made reusable without dependencies on
swing.

I've always thought a style system was a great idea for a RIA
framework, and argued strongly for it when I was working full time in
this space. It is, however, a little static for the examples we're
throwing around here. For example, the alternate row highlighting
that's already in place.

OTOH, one option would be that the renderer uses an expression to
figure out what style to apply to a particular cell/widget: in most
cases, the expression would simply be a style name, but more
complicated expressions could be supported. This way, the graphicly
minded folks could focus on designing the style sheet and the
developers can have easy access to the results.

>> I've started playing with Jeppers as the core of a spreadsheet
>> application (which I'll tell you all more about later), and it's pretty
>> well thought out. It also has a query language with references that
>> you'd want to take a look at. There are still some rough edges but I was
>> able to get a working spreadsheet up in a few hours with cell references
>> and basic calculations. (I tried to write my own expression parser too.
>> It took me 8 hours just to get arithmetic working so I decided to reuse
>> the hard work of others. Ahh, the beauty of open source.
>>

The langauge was one of the goals of my project (not any particular
syntax, but the implementation of a parser that creates compound
functors). My spreadsheet was/is an interesting intermediate
application of the language that looks like it'll be interesting in
its own right. I am considering breaking it out as a separate project
in the desktop space (particularly since I spend a lot more time in
the Desktop area than in the Patterns area, where jga lives).

>> In all of the talk of functors I would suggest you limit your scope. If
>> the functors for tables get too advanced you will eventually either
>> reach a limit where it can't do any more, or else you'll reinvent SQL.
>> Figure out what actual developers will want to use the system for and
>> then design your functor/functional-language system to support that. You
>> may find that reusing an existing language better suits your needs.
>>

This goes back to that 'reuse the hard work of others' thing: I've got
quite a few hours into the functor library already (coming up on three
years since the original BOF that started the jga project, but the
basic ideas had been cooking for several years prior to that). I
think that the work I've done can be applied here.

As far as complexity goes, the idea behind the jga functors is to
provide a reasonably complete set of primitives. None of the functors
themselves are difficult, although it does tend to create a large API
to learn. The parser is intended to address that issue. In code, I
tend to use single functors or simple composites directly, while I
tend to use the parser when using constructor/method calls, or
compound expressions. The parser really comes into play where a markup
language is involved, like the worksheet's written form, or the JDNC
description language.

>> (reinventing LISP, i tell ya!

could very well be: perhaps a strongly typed lisp without the parenthesis.
You know what they say about any reasonably complex system (in your case,
probably well enough to provide the citation).

Dave Hall
http://jga.sf.net/
http://jroller.com/page/dhall

PS: if you haven't looked at it, my worksheet is:

http://jga.sourceforge.net/docs/Hacksheet/HacksheetDemo.shtml

with a sample application:

http://jga.sourceforge.net/docs/Hacksheet/ColorSheet.shtml

Patrick Wright

Dave


> I've always thought a style system was a great idea for a RIA
> framework, and argued strongly for it when I was working full time in
> this space. It is, however, a little static for the examples we're
> throwing around here. For example, the alternate row highlighting
> that's already in place.
>
> OTOH, one option would be that the renderer uses an expression to
> figure out what style to apply to a particular cell/widget: in most
> cases, the expression would simply be a style name, but more
> complicated expressions could be supported. This way, the graphicly
> minded folks could focus on designing the style sheet and the
> developers can have easy access to the results.

There are a number of people who have worked out styling operations for
Swing, at least one of which uses a variation on CSS to do this. I think
there are two good reasons to use it: one is where, as Josh points out,
you want to use a Flyweight pattern to limit the number of styling objects
(fonts, colors, borders) and second, because you can write something that
says, "all jbuttons use Verdana 10pt" and "all default jbuttons glow" (as
in OSX I think).

The downside is getting the association tight enough to be generally
usable. That is, the styling selectors have to be both rich but also
non-ambiguous. On the Flying Saucer project (XML/XHTML/CSS renderer), it
seems at this point that the CSS matching to elements itself is not an
issue, in other words, there is no ambiguity there; there is ambiguity or
decisions related to how layout is interpreted as regards certain style
properties, but that has to do more with the richness and complexity of
layout than it does with styling.

I'm not ready to propose that JDNC include a styling system, but there are
advantages. In jEdit for example (multi-type text editor in Swing), you
can set some styles for buttons, or menus, or etc. that apply across the
board. What is important is that that level of control ("all buttons",
"all menus", "all dialogs") is a common way (IMO) for a user to think
about the problem.

If I understood your comment correctly, yes, there is an issue related to
dynamic styling. In Flying Saucer, the current mechanism for handling
dynamic styling on mouse hovers, for example, is a little complex (because
it's attempt 1 or 2). But I do think the Flyweight pattern is good to keep
in mind here.

The three or four "major" record-based RAD/report tools I've used (Access,
PowerBuilder, Crystal Rep., FileMaker) all let you assign expressions for
display characteristics. I don't recall that they let you do "styling",
although in the GUI you could select more than one column and apply a
style (say, bg color) to all of them.

Meta-features I think are important here:
- expression-based controls on formatting characteristics
- minimal impact on performance if many formatting options are applied
(e.g. "christmas tree" display of NYSE stocks in a table)
- dynamic re-styling on data-bound changes
- predictable/clear/simple interpretation of what expressions mean: order
of evaluation, impact of range functions, cost

> The langauge was one of the goals of my project (not any particular
> syntax, but the implementation of a parser that creates compound
> functors). My spreadsheet was/is an interesting intermediate
> application of the language that looks like it'll be interesting in
> its own right. I am considering breaking it out as a separate project
> in the desktop space (particularly since I spend a lot more time in
> the Desktop area than in the Patterns area, where jga lives).

I did want to say that I think the spreadsheet is a great example use of
JGA, and allows for a good way to play around with the results of your
work.

> This goes back to that 'reuse the hard work of others' thing: I've got
> quite a few hours into the functor library already (coming up on three
> years since the original BOF that started the jga project, but the
> basic ideas had been cooking for several years prior to that). I
> think that the work I've done can be applied here.
>
> As far as complexity goes, the idea behind the jga functors is to
> provide a reasonably complete set of primitives. None of the functors
> themselves are difficult, although it does tend to create a large API
> to learn. The parser is intended to address that issue. In code, I
> tend to use single functors or simple composites directly, while I
> tend to use the parser when using constructor/method calls, or
> compound expressions. The parser really comes into play where a markup
> language is involved, like the worksheet's written form, or the JDNC
> description language.

My guess is there will be a long and hard fight in JDNC over what sort of
expression language to include and what it should be used for.

Another case of: keep the easy operations easy, and the hard stuff
possible. Or: identify the major target uses for expressions in various
contexts, and optimize for that (and for GUI builders). It's Java, so
people can always drop down to code to handle odd cases.

Patrick

---------------------------------------------------------------------
To unsubscribe, e-mail: jdnc-unsubscribe@jdnc.dev.java.net
For additional commands, e-mail: jdnc-help@jdnc.dev.java.net