Community:SummerOfCode08:TSF

My attempts at adding Text Services Framework support to the Mozilla platform as part of Google Summer of Code 08. (wish me luck!)

Status

Now that the foundation work is there, new cool stuff will start to pop up :)

The Tablet Input Panel is working now and the icon is showing up:

This shows that TSF is activated and also Mozilla textboxes (the address bar in this case) are successfully exposed to the system (i.e. the system has access to the contents of these textboxes). --Darchons 14:24, 1 July 2008 (PDT)

The end of week 5, and a majority of the text store (the essential part of a TSF implementation and therefore the project) is implemented, and for the unimplemented methods I have a pretty good idea of how to implement them.

• Mozilla is TSF-aware
• Traditional IME's work (but no visual composition indicator, i.e. underline)

I think I'm ready for my first patch and in terms of completeness my implementation is on par with that of the last patch submitted for the bug a few years ago. In terms of actual functionality my implementation is a lot better because my implementation exposes the actual text boxes in Gecko to TSF while the old patch which did not. --Darchons 08:17, 28 June 2008 (PDT)

This is the end of week 4, (end of "full-time" week 1 for me since I had school and exams the first three weeks :/), and I have made stubs for the implementation, implemented several easy-to-implement methods, and made a basic outline of my design. --Darchons 10:48, 20 June 2008 (PDT)

Outline/Progress

Development

• Startup code
  • nsWindow::nsWindow

• Shutdown code
  • nsWindow::~nsWindow

• Focus/blur code
  • ITextStateObserver::OnFocusChange

• ITextStoreACP implementation (24 out of 26 implemented)
  • AdviseSink
  • UnadviseSink
  • RequestLock
    • addnsWindow input block (not required now but would be nice to have)
  • GetStatus
  • QueryInsert
  • GetSelection
    • add NS_QUERY_SELECTED_RANGE_TEXT event
  • SetSelection
    • add NS_SELECTION_SET event
    • add NS_SELECTION_ADD event
  • GetText
    • (use NS_QUERY_TEXT_CONTENT event)
  • SetText
    • (call SetSelection then InsertTextAtSelection)
    • can we prevent painting during the method?
  • GetFormattedText
    • somehow make use of nsDataObj
  • GetEmbedded
  • QueryInsertEmbedded
  • InsertEmbedded
  • RequestSupportedAttrs
  • RequestAttrsAtPosition
  • RequestAttrsTransitioningAtPosition
  • FindNextAttrTransition
  • RetrieveRequestedAttrs
  • GetEndACP
  • GetActiveView
  • GetACPFromPoint
    • add hit test event
  • GetTextExt
    • NS_QUERY_TEXT_RECT
  • GetScreenExt
    • NS_QUERY_FRAME_RECT
  • GetWnd
  • InsertTextAtSelection
    • NS_TEXT_TEXT
  • InsertEmbeddedAtSelection

• Notification code, ITextStateObserver (7 of 7 implemented)
  • OnAttrsChange (not supported)
  • OnStartEditTransaction (not supported)
  • OnEndEditTransaction (not supported)
  • OnLayoutChange (not used)
  • OnSelectionChange
    • ITextStateObserver::OnSelectionChange
  • OnStatusChange (not supported)
  • OnTextChange
    • ITextStateObserver::OnTextChange

• Display attributes support

• Input scope support
  • -moz-input-scope property

Testing/Debugging

• Writing tests
• Tablet input
• Speech recognition

Thoughts

I was an extension developer for a while before this, but it definitely feels different to be working directly on the Mozilla trunk :) and as with a lots of things there are some goods and bads

One major goodness is how easy it is to do some of the stuff that I was doing.

Extreme example: Getting a window handle (HWND) to the Firefox window. In my extension I had to, (there are probably easier ways to do this that I haven't found)

• Start off with a nsIDOMNode
• Check for accessible attribute on the node
• If exists, get the nsIAccessible
  • QI to nsIAccessNode
• If doesn't exist, get the accessibility service ("@mozilla.org/accessibilityService;1")
  • QI for nsIAccessibleRetrieval
  • Call nsIAccessibleRetrieval::getAccessibleFor to get a nsIAccessible
  • QI to nsIAccessNode
• Call nsIAccessNode::GetAccessibleDocument to get a nsIAccessibleDocument
• Call GetAccessibleDocument::GetWindowHandle to get the HWND

It's not very pretty and you have to hope that accessibility is working (and included in the build, which thankfully is the default)

Now, for what I'm doing now,

• Start off with a pointer to nsWindow
• Call nsWindow::GetWindowHandle to get the HWND

The difference between the two is pretty big.

Now one bad thing is that in my extension I could pretty much use everything that the platform offers, as long as it accomplishes my goals.

But right now I'm working under the widget component which is lower-level to content, layout, etc. And I cannot directly use most of the stuff from these high-level components from widget otherwise that would break the abstraction. So it feels more constricted in a way than when writing an extension. But there are alternatives and these shouldn't cause problems.

Quirks

Firefox 3 is crashing on me constantly, and most of them aren't even getting caught by the crash report thingy and they are getting caught be the crash report thingy now (thx Neatnate) but FF3 is still crashing a lot :(

Well turns out this is what's bugging me the whole time (no pun intended) bug 432467 (thanks Ted!)

Installing FF3.0.1 build 1 seems to have fixed the problem :)

Praises

I'm using my laptop, which has a 12.1" screen, for this project, and it definitely makes it a lot easier to code when I hook it up to the main desktop's 22" LCD.