Improving Code Assist with JSDT and NJSDoc

[paulvi]

For #7 we should try NJSDoc, that actually depends on JSDT.

https://bitbucket.org/nexj/njsdoc

http://imgur.com/a/bT4Cd

[paulvi]

Currently there is problem when installing in Eclipse for JEE or STS (that both have latest JSDT):

  Details:
  Your original request has been modified.
    "JavaScript Development Tools " is already installed, so an update will be performed instead.
  Cannot complete the install because of a conflicting dependency.
    Software being installed: JavaScript Development Tools  1.5.0.201304171754 (org.eclipse.wst.jsdt.feature.feature.group 1.5.0.201304171754)
    Software currently installed: Eclipse IDE for Java EE Developers 1.5.2.20130211-1820 (epp.package.jee 1.5.2.20130211-1820)
    Only one of the following can be installed at once: 
      JavaScript Development Tools Core 1.1.201.v201203071941 (org.eclipse.wst.jsdt.core 1.1.201.v201203071941)
      JavaScript Development Tools Core 1.1.202.v201208171701 (org.eclipse.wst.jsdt.core 1.1.202.v201208171701)
      org.eclipse.wst.jsdt.core 2.0.0.201304171754
    Cannot satisfy dependency:
      From: Eclipse IDE for Java EE Developers 1.5.2.20130211-1820 (epp.package.jee 1.5.2.20130211-1820)
      To: org.eclipse.epp.package.jee.feature.feature.group [1.5.2.20130211-1820]
    Cannot satisfy dependency:
      From: Java EE IDE Feature 1.5.2.20130211-1820 (org.eclipse.epp.package.jee.feature.feature.group 1.5.2.20130211-1820)
      To: org.eclipse.jst.ws.jaxws.dom.feature.feature.group 0.0.0
    Cannot satisfy dependency:
      From: Eclipse Java EE Developer Tools 1.0.500.v201205170400 (org.eclipse.jst.jee.ui 1.0.500.v201205170400)
      To: bundle org.eclipse.wst.web.ui [1.1.0,1.2.0)
    Cannot satisfy dependency:
      From: JAX-WS DOM Tools 1.0.200.v201109042201-5-F8NAkF7BB7U8PEK8K (org.eclipse.jst.ws.jaxws.dom.feature.feature.group 1.0.200.v201109042201-5-F8NAkF7BB7U8PEK8K)
      To: org.eclipse.jst.jee.ui 1.0.100
    Cannot satisfy dependency:
      From: JavaScript Development Tools  1.5.0.201304171754 (org.eclipse.wst.jsdt.feature.feature.group 1.5.0.201304171754)
      To: org.eclipse.wst.jsdt.core [2.0.0.201304171754]
    Cannot satisfy dependency:
      From: Eclipse JavaScript Development Tools 1.1.201.v201204101526 (org.eclipse.wst.jsdt.ui 1.1.201.v201204101526)
      To: bundle org.eclipse.wst.jsdt.core [1.1.101,2.0.0)
    Cannot satisfy dependency:
      From: Eclipse JavaScript Development Tools 1.1.202.v201208171701 (org.eclipse.wst.jsdt.ui 1.1.202.v201208171701)
      To: bundle org.eclipse.wst.jsdt.core [1.1.101,2.0.0)
    Cannot satisfy dependency:
      From: Eclipse Web Developer Tools 1.1.600.v201109220400 (org.eclipse.wst.web.ui 1.1.600.v201109220400)
      To: bundle org.eclipse.wst.jsdt.ui [1.0.0,2.0.0)

In Aptana, installation goes, finishes with error (see below), but after restart both JSDT & NJSDoc are in.

[johnpeb]

When installing JSDT+NJSDoc it's important that JSDT from the update site is used: https://bitbucket.org/nexj/updatesite/raw/default . It might be necessary to first uninstall existing JSDT related plugins. If all else fails try install into Eclipse Classic.

I haven't done testing of compatibility as JSDT+NJSDoc is only used internally so far.

[paulvi]

That is unacceptable, because this way is limiting Eclipse flavors where Nodeclipse can be installed on, or makes installation complicated and not compatible with other tools.
This way can be only used during development, but will not go into 0.4.

Can you take the latest JSDT and fix it every major release or contribute to JSDT project?

[johnpeb]

Of course I would like to contribute this to the JSDT project. However

• JSDT has no active development
• Even if they were willing and able to take my contribution there is a lot of work before the rough edges are worked out

In the short term if my fork of JSDT that uses NJSDoc will be integrated with Nodeclipse then a lot of renaming will have to occur so that it can coexist with JSDT installs

[paulvi]

Other option may be to rename your fork, for example to JSDT2. In such a way, that JSDT and JSDT2 can coexist.
I don't know if it is really feasible. Can you estimate?

[johnpeb]

JSDT2 sounds like a good name. I will send one more email to the JSDT (wtp) mailing list to see if they are interested in merging back in. Failing that we can rename to JSDT2. A problem is I won't have time to work on this until mid or late may.

[paulvi]

OK. Meanwhile we can work on this as option, that is only available on Eclipse without JSDT or with old JSDT.
(That sounds no so good, but we can attach word 'experimental' for that)
By the way, can there be problem if, for example, Eclipse 3.6 user gets JSDT updated this way?

[tomotaro1065]

Hello John,

I cloned your nexj/njsdoc-build, nexj/webtools.jsdt.core and nexj/updatesite .
Then I tried to build them, but I got an error that said "Unable to find feature "org.eclipse.license" with version in range [1.0.0,1.0.1)." .
I also cloned standard JSDT from GitHub, and I got the same error.

What is the cause of this error?
What should I do to solve this problem?

Best Regards.

[johnpeb]

I found that feature in this repo: git://git.eclipse.org/gitroot/webtools/webtools.releng.git

[tomotaro1065]

Hi John,

Thanks. I cloned that repository. But I couldn't understand how to use those files.
Is it a tool that is mentioned here ?
I installed the tools from http://download.eclipse.org/webtools/releng/repository into Juno-RCP.
And select [Releng]-[Migrate License Support] menu.
Following messages appeared on the console.

Total number of features found: 3
     Total number of primary features found: 2
     Total number of source features found: 1


The following feature(s) were found ok, and not modified
    Count: 3
    org.eclipse.wst.jsdt.feature
    org.eclipse.wst.jsdt.feature[sourceTemplateFeature]
    org.eclipse.wst.jsdt_sdk.feature


Elapsed processing time : 48 ms

It seems nothing happens. I have still the same error after that when building JSDT.

[johnpeb]

Just for the record, lets continue this on the wtp-dev mailing list at least until it gets to NJSDoc specific stuff: http://dev.eclipse.org/mhonarc/lists/wtp-dev/msg08656.html

[paulvi]

Quick link is this https://dev.eclipse.org/mailman/listinfo/wtp-dev

[paulvi]

Changed to 0.5, we can't possibly do it within 0.4 cycle

[tomotaro1065]

I succeeded to build JSDT.
http://dev.eclipse.org/mhonarc/lists/wtp-dev/msg08663.html

[paulvi]

I browsed wtp-dev mailing list, some main points:

• good chance for @johnpeb changes to be accepted by Eclipse
  Wayne Beaton (Director of Open Source Projects at the Eclipse) has liked quality of changes (http://dev.eclipse.org/mhonarc/lists/wtp-dev/msg08642.html),
• however, it is not clear when, how or relation to VJET (Talk to Justin Early?)
• not clear when Improving Code Assist with JSDT and NJSDoc #16 (this) will be possible.

So though pure development work is not so big (around a week, as estimated by John),
I guess it will take a month or two just to settle down organizational topics ( @johnpeb Are you more optimistic? )

Then there is date May 24/2013 - Deadline to submit IP Logs for Kepler releases, actually 4.3RC release
http://wiki.eclipse.org/Kepler

That means chance for NJSDoc support within 4.3 Kepler are slim.

@tomotaro1065 Do you feel like building JSDT ? I saw Mike Milinkovich is quite serious about Eclipse trademark
http://dev.eclipse.org/mhonarc/lists/wtp-dev/msg08639.html

I guess John now knows better about possible way to release JSDT2 in nice way.
@johnpeb Can you give advice for next steps?

[johnpeb]

Hi guys, currently the maintainer of JSDT at IBM is evaluating the NJSDoc version of JSDT.. I'll probably hear in the next few days what the outcome is and whether there will be a fork or not. Hopefully not. I'll update here when I hear. By the way, below is the text version of some slides I wrote up on NJSDoc+JSDT


Background

* High quality content assist requires knowing the API that code makes available to you 
* Traditional techniques use static analysis – JSDoc and variants. 
* Other approaches (like VJET) require defining the APIs manually beforehand  
       which works well if your API is fixed 

Introduction to NJSDoc 

* NJSDoc does dynamic analysis of the code – it executes the 
code in your project to find out what the code defines. 
* It tracks declarations and assignments to correlate code 
locations and documentation to nodes in the runtime 
structure 
* It further analyzes the runtime structure to infer classes and 
class members by walking the prototype chains 
* Benefits 
  * Accurate 
  * Very fast 


* Downsides 
  * Complicated to set up NJSDoc recipes 
  * For structures to be documented they must be exposed globally 

Approach – extending JSDT 
* Use NJSDoc for global resolution for content assist 
* Use JSDT for local resolution for content assist 
* Use JSHint-eclipse for problem checking 
  * Provides excellent syntax and code quality feedback 
  * JavaScript is a dynamic language and I believe problem checking 
globals suffers seriously diminishing returns 


Future work 

* In theory NJSDoc could be integrated with global JSDT 
problem checking but doing so would require large 
improvements to JSDT problem checking and would still not 
catch all bugs 
* Add support for node JS style module definitions 
* Implement support for inferring types of module imports in 
AMD modules 
* Remove more JDT specific code 



[paulvi]

Now wit NTS we can include any version of plugins.

However I strongly suggest that we support all Eclipse distribution, and both Code Assist and Code Assist with NJSDoc should be default but optional.

[paulvi]

Eclipse Bug 406575 - Investigate merging NJSDoc support into JSDT

[paulvi]

@johnpeb Hi John

I was trying to install NJSDT+NJSDoc in Eclipse for Tester (the lightest Eclipse and without JSDT).
I tried 3 times, and couldn't finish, because of network.

Could you consider host NJSDT+NJSDoc on some more cloud-ready platform like GitHub.
And in particular in Nodeclipse update site. What do you think?

[johnpeb]

Hi Paul, does the issue persist? Could it be a china firewall issue? Can you access those URLs directly? Here's a direct link: https://bitbucket.org/nexj/updatesite/downloads/com.nexj.njsdoc_0.0.6.201304171754.jar

I'm hesitant to move the update site as my co-workers use it and I need to control the release schedule.

[paulvi]

I couldn't update from home, but I succeeded from office. I think it is more related to network latency. So on slow Internet connection, resource that are far away (>50ms ping), some website & resources occasionally report that they are not available, because of timeout settings.

No need to move, just copy. We can redistribute.
When it will be from one site, it should be more stable for Nodeclipse users.

Now some Nodeclipse users needs to install JSDT see #46 , so I propose to default them to NJSDT.
It is stable, isn't it?

I will also update Enide soon.http://marketplace.eclipse.org/content/enide-eclipse-nodejs-ide

[johnpeb]

Yes NJSDT is stable with a couple bug fixes and performance optimizations over standard JSDT even if the NJSDoc features aren't used.

The other complication with also hosting on the Nodeclipse site is the trademark issue they mentioned on the WTP-dev list. I have been traveling and on vacation last week and part of the week before otherwise I would have spoken to the IBM guys again. I hopefully will this week.

[paulvi]

Will it be not complicated to have NJSDT and NJSDoc as separate solution (that are based on Eclipse JSDT) with its own name? Would it be OK with Eclipse, I wonder?

Meanwhile, as this topic #16 is about improving code assist in Eclipse for Node.js developers, I would stick to use solution that is closer to solve problem. That is NJSDT and NJSDoc.

It will be default in Enide (when JSDT is not installed, because of version numbers).
However there is latency problem, as I see it GitHub is quicker that BitBucket, however simplistic that sounds.

[paulvi]

@johnpeb Will there be updates for NJSDT? Should this issue be closed?

[paulvi]

I guess Eclipse Bug 406575 - Investigate merging NJSDoc support into JSDT resolution was anti-encouraging to continue the project