Can Piccolo Take Fluency's Weight?
What is piccolo?
From http://www.cs.umd.edu/hcil/piccolo/index.shtml
A revolutionary way to create robust, full-featured graphical applications in Java and C#, with striking visual effects such as zooming, animation and multiple representations.
Piccolo is a toolkit that supports the development of 2D structured graphics programs, in general, and Zoomable User Interfaces (ZUIs), in particular. A ZUI is a new kind of interface that presents a huge canvas of information on a traditional computer display by letting the user smoothly zoom in, to get more detailed information, and zoom out for an overview. We use a "scene-graph" model that is common to 3D environments. Basically, this means that Piccolo maintains a hierarchal structure of objects and cameras, allowing the application developer to orient, group and manipulate objects in
meaningful ways.Why use Piccolo? It will allow you to build structured graphical applications without worrying so much about the low level details. The infrastructure provides efficient repainting of the screen, bounds management, event handling and dispatch, picking (determining which visual object the mouse is over), animation, layout, and more. Normally, you would have to write all of this code from scratch. Additionally, if you want to build an application with zooming, that's built right into the framework too.
What exactly is it? Piccolo is a layer built on top of a lower level graphics API. There are currently three versions of the toolkit: Piccolo.Java, Piccolo.NET and PocketPiccolo.NET (for the .NET Compact Framework). The java version is built on Java 2 and relies on the Java2D API to do its graphics rendering. The .NET version is built on the .NET Framework and relies on the GDI+ API to do its graphics rendering. This makes it easy for Java and C# programmers, even those targeting PDAs, to build their own animated graphical applications. And best of all, Piccolo is free and open source!
Definitions
First, let's define a few terms so that everyone in Fluency's builderland is on the same page.
| term | definition |
|---|---|
| Author | someone who builds visual interfaces (GUIs). |
| Builder | the GUI that an author would use to build fluent interfaces. |
| Builder Editor | a view (plus tools/controls) in which information that the author needs to build a user interface is presented. Fluency presently only has 2 such Builder Editors (the Layout Editor and the Linkage Editor). |
| Layout Editor | the place in the Builder where visual widgets are dropped onto a canvas, and where their visual properties are changed. 'Visual properties' include background color, size, location, etc. |
| Linkage Editor | the place in the Builder where the widget linkages (the real application logic) is created. This is also where non-visual widgets are created. They are added to some set of visual widgets which are automagically imported from the Layout Editor. The non-visual widgets interact with the visual widgets to create the fluent GUI being built. |
Note
One possible use for a framework like piccolo is to support Fluency's current linkage graph (thus replacing the GEF) plus anything drawn on the glasspane in the layout editor (for example: a fancy way to set visual widget properties). (Mitch has started working on a non-eclipse (actually, just non-GEF so far) version of the fluency builder using the piccolo framework. Click here to run it from the web.) One possible (longer-term?) goal might be to replace eclipse entirely, as Mark has asked. But until we know more that seems like a more major step than simply replacing the Linkage Editor.
Exploratory (no more than a week, if possible)
Part A
How hard is it to make piccolo animations show up on a JPanel (or glasspane)? If not out of the box, can we make this happen the same way Jazz does this?
(Note: Jazz was the predecessor of Piccolo; it included Swing widgets directly. You can learn about it on the same UMD website as Piccolo's above.)
Part B
How hard is it to intercept events (mouse and keyboard) in a piccolo interface?
Part C
How hard is it to connect both of the current builder Editors so that they mirror each other? (Note that that only needs them to have the same MVC Model. In practice though, there might be a certain amount of fighting with GEF and eclipse. Also: it would mean making some decisions about where the various widget properties (for visual and non-visual widgets alike) would be stored. One perhaps reasonable place is to store everything in compartmentalized chunks on the widgets represented in the backend.)
Part D
How hard is it to connect that (builder) MVC Model to the (backend) Model with events (or signals or whatever we're going to call them) passing between the two easily?
Deliverables (no more than 2 weeks, if possible)
Part 1: Widget Linkage
Authors should be able to hover over a widget and see what it can emit (as events or ships, or whatever they're to be called). An author should also be able to grab one such potential signal and connect it to another widget. That target widget should display the slots that that particular signal can connect to, and nothing more, thus making the author's life easier. (Note: Fluency has never supported many-many interwidget connections, but there's no reason in principle why we couldn't have them one day, as the design document outlines.)
Part 2: Near-Widget Pop-Up Properties
Authors should have some sort of (perhaps circular) fisheye-like effect to set a widget's properties. When an author hovers over a widget in the layout editor, the properties should expand out around the widget in some compact manner. Then when an author hovers over a particular one of those properties, it should expand with some sort of "picker" or "setter" of available values. Most likely each of the properties will have a different picker though some would be similar (ex: booleans – hidden, active, etc).
Part 3: Compound Actions
Authors should be able to create compound actions (or their nautical equivalent) in the linkage editor. Perhaps this might mean dragging actions from various widgets and placing them in some sort of container to thus make a sequenced compound action. It would be a plus if authors can also rearrange such inserted actions (i.e. it would be nice if the action list supported insert and delete). Authors also need some way to specify how to trigger execution of such a compound action (which then executes each of its actions in order) when some signal comes in. (But "comes in" where? Presumably, compound actions would only now exist as dynamically created Fluency actions, since with the redesign actions can no longer live in pipes and thus must always be attached to some widget. This means that Fluency itself needs a manipulable (non-visual) widget in the linkage editor for the author to connect pipes to (or, in the new metaphor, shipping lanes to.)
Part 4: Clutter Management
Authors should be able to select chunks of the linkage and temporarily "minimize" it away so that it doesn't clutter up the space and thus distract them from their current task or thought process. Authors should be able to both "peek" into a minimized blob of linkage or expand one out again to do some more work on it.
Part 5: Layout and Containment (perhaps extra credit? might be way hard...)
If we were to try to get out of the eclipse straightjacket, we'd also have to replace the current Layout Editor with a (perhaps) piccolo-style layout editor. That would mean having to write at least one of our own layout managers. How hard is that? Also: what might a shift to a piccolo layout editor mean for containment?
End Point
By the end of this 'assignment' it sure would be nice to have some idea of how easy or hard this list of problems might be in reality. Ideally, it would be great to have some of them actually solved in the next 3 weeks.
Comments (17)
Oct 08, 2006
Mitch McCracken says:
an important thing is being able to communicate the backend widget information i...an important thing is being able to communicate the backend widget information into the GUI. the backend needs a good way of storing properties and actions that the GUI can import. after we knock that out, we can figure out the best way to graphically display the information.
Oct 08, 2006
Nate Johnson says:
i would hope the backend would store the model, and the frontend would just disp...i would hope the backend would store the model, and the frontend would just display that model visually, rather than import it somehow. maybe that's what you mean, but i just wanted to make sure.
-nate
Oct 08, 2006
whoknows says:
we've had various arguments over the years by certain purists in the class who'v...we've had various arguments over the years by certain purists in the class who've felt that a model shouldn't contain any visual information. strictly, speaking, with MVC it really shouldn't (instead, it's each View's job). however, there's theory and then there's reality.
i've always felt (and argued) that there's nothing wrong with each widget in the backend model having a collection of view properties.
(note: this doesn't mean that any Widget object needs to know about such properties (a common point of contention in the past) but rather just that Fluency can find the right view's properties for any widget on request from the frontend, which is a different thing.)
each view would need its own properties (for example, the position and size of a button widget in the layout view could be completely different from the same button widget's position and size in the linkage view---or any other future view we might choose to create later). such properties could then be saved and restored generically together with the rest of the state, if any, of each widget. further, all that persistence could happen on the backend inside our usual persistence subsystem. instead, what we've had for years, is a hardcoded, handrolled, builder (every version of Fluency so far anyway...) and it's usually not even persistent (even back in the day when persistence worked on the backend).
a view could query the (frontend) model (which would be populated by the backend model, say) to get a set of properties for each widget appropriate to that particular view (no view need be nor should be aware that there are other views, just as no widget need be aware that any other widgets exist). some widgets would display in only one view (ex: all non-visual widgets in the linkage view), others might appear in more than one (ex: any visual widget), but their actual appearances would depend on those view-specific properties.
for some years now i've argued for that design. hasn't happened yet. hopefully this is the year it'll take hold. or maybe you guys will think of something even more clever and clean.
-gjer
Oct 08, 2006
Russell Duhon says:
I don't think there's any problem with model objects having view properties. It ...I don't think there's any problem with model objects having view properties. It might be considered problematic in an MVC design, but we don't have an MVC design. We have towering widgets that integrate all three levels and a need to organize those towers both visually and logically in visual editors. Given that what we're trying to organize necessarily incorporates model, view, and controller in its identity, maintaining MVC separation has issues.
MVC is nice because it is one successful approach to reducing coupling and increasing maintainability. Fluency is a paean to another way of doing things, so going a non-MVC direction is more than understandable.
And of course, ultimately all model objects need are properties. That another object interprets those properties to have visual significance is none of the model object's business, particularly if those properties are generic rather than specified.
Oct 08, 2006
Russell Duhon says:
My thoughts on the assignment: Part 2 is nice to have, but hardly essential. A l...My thoughts on the assignment:
Part 2 is nice to have, but hardly essential. A list of properties is definitely useable, if mildly clunky, and used by even some applications considered extremely useable (my favorite examples is omnigraffle). Large other parts of fluency are, however, not useable. The focus should be on them.
Part 3 comes out of nowhere. I have no idea what a use case is for this. We already have a way to make certain some actions trigger further actions – connect outputs to inputs. Beyond that weak notion, the notion of 'ordering' the execution of actions almost doesn't make sense due to the fluency clock. On each clock tick, actions with the appropriate inputs are executed, and actions without them aren't. Why do we need this?
We definitely want something like part 4. However, I think a better approach will be an assumption that anything (in linkage) not 'near' what the author is currently worked with should be kept out of the way, and only made more noticeable as the author chooses it should be. At any given time, most of linkage interface will be completely irrelevant to what an author is doing. Best for it to stay out of his or her way automatically.
Regarding Part A, PCanvas (how one embeds a Piccolo view in a swing application) is a java.awt.Container, so it should be possible to add swing components to it like any other Container. checks yes, that seems to be the case. I added the following line to GUI.java in testgui, and it continued to work much as before: layoutApplication.setContentPane(new PCanvas());
Part B looks promising. There is support for a wide variety of listeners, particularly all sorts of mouse interaction. Support for keyboard interaction is a little harder to fathom, but the PSWTKeyEvent suggests a good deal of possibility; since Piccolo stuff inherits from Swing, it should be able to do any contortions swing can do, too.
In Part C, I'm not sure what is meant by mirroring each other.
Part D seems to be the crux of things.
Oct 08, 2006
Nate Johnson says:
for part 3, what i was thinking is a bit like a macro. if there is something com...for part 3, what i was thinking is a bit like a macro. if there is something complex that the author wants to happen and happen often (by many different events), then extracting that action out into a compound action would be nice. it could still run off of a central clock. the ordering bit would be nice for the author to, well, reorder the actions in the macro. i'll try to give an example: say an author wants to get three text fields, set them on some log widget, and execute a save action. that's 7 actions. they could be executed by any action in a fluent application and it would be nice to not have to recontruct that all over the place in the linkage (as it will get hairy – though part 4 helps here).
-nate
Oct 08, 2006
Russell Duhon says:
Assuming I'm reading you write, I'm not sure the compound action notion provides...Assuming I'm reading you write, I'm not sure the compound action notion provides significant, if any, benefits, as we talked about in our IM conversation.
Here are some numbers related to your example that occurred to me as I was walking my dog before going to sleep:
Without compound actions we need to draw three pipes (from one triggering event) to each of the three 'get text' actions, three pipes from from each of the 'this is the text' output events to this log widget that takes three inputs or something, and one pipe from whereever is triggering this save action (maybe we dual-purpose one of the 'this is the text' events?). That's a total of seven non-reduceable, individually quick and easy tasks.
With compound actions (assuming we've set up the compound action in advance, though in reality this would add to the time), we need to specify which event will trigger the compound action, which three text widgets are being used, and which log widget is being used. That's a total of five non-reduceable, individually quick and easy tasks.
So there's a small benefit to having the capability, assuming there are enough repeat uses of a pattern for an author that the time it takes to set up the compound action is negligible. However, there's also a cost of making it non-obvious how the links work between different widgets. This effectively hides some of the pipes flowing between certain widgets, making it impossible to determine the operation of parts of the application by tracing pipes, since the real pipes are hidden within the compound action.
Oct 08, 2006
Nate Johnson says:
i think a lot of this comes down to a really usable builder. if we had that, may...i think a lot of this comes down to a really usable builder. if we had that, maybe we don't need compond actions at all (gregory, am i forgetting something here?) – that is, if the interface rocks, things like compound actions might include dragging a box around a bunch of actions and dragging it to an event which executes it all... i don't know...
nate
Oct 09, 2006
whoknows says:
right at the moment all i care about is basic functionality. it would be a first...right at the moment all i care about is basic functionality. it would be a first in fluency just to have that, and have it be reliable and easily extensible. i don't much care about compound actions, per se, except in so far as in the earlier design they came nearly 'for free' (with the composite pattern) and they allowed fluency to (theoretically, at least, i don't know what actually made it into the codebase) to centralize its own actions, and so made maintenance and persistence of the linkage graphs (there are actually two: one the author 'sees' in the linkage view and another that fluency alone sees; the idea was that the author might not want to be bothered with certain kinds of links (ex: pipes connecting to pipes to accomplish some task more complex than could be done by a generic pipe) but that fluency would have to keep track of). with the backend redesign though that could potentially change. also: even though i'd planned for pipes to connect to pipes (and pipes to be many-many etc) so far that's not supported by the codebase, old or new.
having said all that, though, i still can't see any clean way around making fluency a widget (as it is now) and attaching its own linkage-maintenance and linkage-persistence actions to it. perhaps there's some way around that, but i'm not sure that it would be better. so far, that still seems both clean and elegant to me. in the redesign, the fluency widget would still be there but it's actions (so far) can't be dynamic, or compound, or even persisted, since they're (currently) just wrappers around pre-written methods. that seems limiting to me, but we'll see how it goes.
-gjer.
Oct 09, 2006
Evan D. says:
Just some more hypercritical design thoughts... =) This all sounds fine to me ex...Just some more hypercritical design thoughts... =)
This all sounds fine to me except for the description of Part 2: Near-Widget Pop-Up Properties...I really dislike the idea of anything "popping up" on a mere mouseover, especially an interactive menu. I think this should be relegated to something like a right-click.
Mice have buttons...if I want to interact with an object, I will click on it, because the buttons are the mouse's canonical/assumed interface for interaction with on-screen objects. When I move the mouse, it should be for positioning the cursor over a given object so that I may interact with it using the buttons.
If the program creates menus on mouseover events, I can see that getting very annoying - if I want to click on the menu bar, for instance, I'll find myself moving my mouse around the edges of the screen so that I don't get all those irritating (and potentially slowdown-causing) popup menus when I mouse over the widgets in my design window.
Oct 09, 2006
whoknows says:
what fluency becomes is now largely in your hands. remember though that you aren...what fluency becomes is now largely in your hands. remember though that you aren't the ultimate users. it's what they want that will really matter. for the moment, we have no ultimate users (until we can entice someone like rick to come back) so do what you want. just do it in such a way that it's backoutable. this relates to the business russell alluded to above about property sheets too, which i chose not to comment on earlier. when we started, property sheets were a natural idea but i always disliked them (i dislike them in all program i have to use that have them, not just in fluency). rick disliked them too. however we did it because 1/ it was easy for programmers to imagine and to do and 2/ it was always to be 'temporary'. well, the years passed and, whaddaya know?, the 'temporary' became 'permanent', just as unthinakbly unalterable as in any other ide. this isn't to say that your point about mouse handling is wrong, or that russell's point about having more serious problems to fix first is wrong. just try to be aware that what programmers like and find natural and easy (both to do and to use) aren't necessarily what authors will like and find natural.
-gjer
Oct 09, 2006
Chathura Herath says:
IMO here are the basic "high level problems" that we face with the visual subpro...IMO here are the basic "high level problems" that we face with the visual subprojects of fluency.
1) Come up with the Fluency visual editor/builder that could allow basic editor/ linkage and composition facility for the visual widgets.
2) How to handle the event filtering in the fluency builder(current solution:event channel)
3) How to handle the event interception and communicating with backend in the runtime(i.e event to ship bridging, etc)
Other than those i belive all the rest of the things are secondary or nice to have features built on top of it. Even the persistance could be considered to be not core, in the context.
The million doller question is, is it possible for us to get a comparative advantage in solving above problems if we use Piccolo. Especially the event handeling problems.
It may have implemented some cool features like ZUI, default event behaviouts, layout managers, etc, from what i have read so far it has the almost very similar UI building API and a similar event handling framework to that of Swing. From first glance it will not make our lives easier w.r.t. the above 3 problems. So we ll have to solve those very same problems that we solved in the past with glass panes and event chanels with this library as well. But shouldn't be too complicated.
Once we are done with that we could use all the claimed cool features to make our lives easier.
IMHO We should get the basic functionality right before we could add the additional features. Trying to think of the not so fundermental features initially will add lot of unnecessary complications to the code.
Other positive points even if we are not going to use it.
They have done the containier management so the i believe they have the event inteception and filtering mechanism already in place.
Also they have claimed to solve the problem of event management including keyboard events. If we are not gonna adopt it may be it would be useful to have look at how they did it, because almost in every class I ve heard complains of how tedious it is to manage focus and thus keyboard events. It might give us an idea of how we could do some of the stuff we wanna do.
My two cents..
Oct 09, 2006
whoknows says:
in case it's helpful on the eventmanagement front, here an extract of something ...in case it's helpful on the event-management front, here an extract of something alek posted on our private fluency list a few days ago. essentially, someone has hacked up an swt-emulator for eclipse to use actual swing widgets. he had to wrestle bigtime with porting event management between both systems.
http://www.infoq.com/news/swt-swing
(...) We asked Deckers about what had been the most difficult areas in
bridging the two API's:
Oct 09, 2006
Russell Duhon says:
nods Those are reasons I'm fairly agnostic towards using Piccolo for layout. I d...nods
Those are reasons I'm fairly agnostic towards using Piccolo for layout. I don't think it would add very much value there, anyways. I'm interested in Piccolo for linkage.
Oct 09, 2006
Vivek S. Thakre says:
In the application created by Mitch using piccolo, if we try to drag the compone...In the application created by Mitch using piccolo, if we try to drag the components out of the parent container(either in linkage or layout), it disaaperars and then it cant be drgged inside the ontainer again.will try to make the appropriate changes so that the components cant be dragged out of the boundaries of the parent container.
Oct 09, 2006
Jaliya Ekanayake says:
Why we need piccolo? May be a stupid question; why are we still going here and t...Why we need piccolo?
May be a stupid question; why are we still going here and there with this project? My idea is that last week we had some goal and if we divide the tasks between teams today and start implementing them at least we will be able to get something within few weeks.
I just played with piccolo and my idea is that we can implement the UI Editor and Linkage Editor using piccolo but again it will be a kind of re-implementing something that is working.
Next stupid question: what happen to our MP3 Sorter?
-Jaliya
Oct 09, 2006
whoknows says:
these are good questions for you guys to argue about. you should argue. just two...these are good questions for you guys to argue about. you should argue.
just two reminder notes from me: first, the business about the mp3 sorter: it hasn't gone anywhere. team 1 argued (and we agreed, after a few weeks of fighting) that it should wait until the refactor was done. they said that when the refactor is done it would be pretty easy to make the mp3 sorter. they also said that if we tried to do it with fluency in its current state we likely wouldn't succeed. we all agreed with them (eventually). remember those long-gone days of a couple weeks ago?
essentially the same argument was made about refactoring the linkage editor (at least) of the builder. the argument was made again by team 1 and supported by at least russell in team -pi. when nate and i saw that mitch made good progress on linkage view using piccolo in just two days we thought it worth investigating whether it's the way to go for us. hence the assignment. (notice it's title: "Can Piccolo Take Fluency's Weight?". the answer to that question is unknown at present.) within a week we should have a good idea if it can be easy or not. several people seem to think it'll be easy. there was much boastage in class today to that effect. we'll see
-gjer