NDEx Release Notes - Q1-2018 Update (v2.2.1)
February 1st, 2018
The NDEx Q1-2018 Update brings changes and new features to improve the end-user experience while optimizing performance and scalability. All new features and improvements in NDEx v2.2.1 are deployed and available only on the NDEx Public Server. No new installation bundle will be released in Q1-2018.
This section provides information about the new features introduced with the Q1-2018 Update. Please review this document carefully prior to using NDEx 2.2.1 for your work.
- Open in Cytoscape - Beginning with v2.2.1, users can open an NDEx network in Cytoscape directly from the NDEx web User Interface. This feature is triggered by a new "Cytoscape button" available in the network view page. Due to technical limitations, right now this feature can only be used if the following conditions are met:
- Cytoscape 3.6.0 or higher must be installed and running on your computer and have the CyNDEx-2 app installed
*** Note: starting with Cytoscape 3.6.1, CyNDEx-2 will become a pre-installed, core app. ***
- The network to be opened in Cytoscape must be either Public or Private and have the Sharable URL enabled
- @context Editor - NDEx 2.2.1 brings back the possibility to control the namespaces associated to any network. The @context Editor can be accessed from the network page and dinamically adapts to the user's privilege level for that specific network. Namespaces can always be viewed but can be added or modified only if the user has "Edit" or "Admin" privileges on the network. Defining namespaces using the @context Editor (or other method) is important as it will make possible for the NDEx UI to use the identifiers specified in the network (represents, alias and other network attributes) and automatically generate URL to point to relevant external resources. Defining namespaces for a network is not mandatory but highly recommended.
- Expanded Network Indexing Policy - In NDEx v2.2.1, the SOLR indexing policy has been modified to optimize server performance, relevance of search results and system scalability. There are now 3 tiers of indexing complexity the users can choose from:
- Tier 0 (T0)
- This is the default indexing behavior and applies to all PRIVATE and PUBLIC networks written to NDEx regardless of their origin (file loading, script, cytoscape)
- T0 corresponds to the PRIVATE (Not Searchable) or PUBLIC (Not Searchable) visibility options currently available in the NDEx UI
- Under T0, the only parameter that is indexed is the network's UUID
- Tier 1 (T1)
- This is the extended indexing behavior and can be applied to any PRIVATE or PUBLIC networks already existing in NDEx
- T1 corresponds to the PRIVATE or PUBLIC visibility options currently available in the NDEx UI
- T1 can only be selected manually in the NDEx UI, either on a per-network basis or as a bulk action
- Under T1, the following network attributes are indexed for search: UUID, name, description, reference, disease, author, labels, tissue, methods, rightsHolder, rights, organism, networkType
- Under T1, a new add-on option will become available via a selectable checkbox in the network property editor to optionally add "full node indexing". If selected, the network will be indexed according to the Tier 2 specs listed below.
- Tier 2 (T2)
- This is the most comprehensive indexing tier
- Full indexing is only available on a per-network basis via a selectable checkbox in the network property editor; this option is not available as a bulk action
- By default, the "Full Index" checkbox is NOT selected
- When selected, in addition to the network attributes indexed under T1, the following node attributes will also be indexed for search: nodeName, represents, alias
- This feature uses a lot of resources and should be used sparingly
- "Certified" and "Pre-certified" networks are automatically indexed under T2
Web User Interface Improvements
In NDEx 2.2.1, the web UI includes several improvements to the general look and feel; the landing page has been decluttered to focus on the most important components: the search bar and featured collections. In addition, the landing page also shows the current NDEx version and provides direct access to the latest Release Notes.
The network view page has new embedded documentation in the form of tooltips that can be activated by hovering on buttons and other UI elements. Tooltips also provide explanatory messages in cases where a certain function is not available to the user.
Finally, the network viewer has been tweaked to render Cytoscape visual styles with higher fidelity when using discrete mappings, continous mappings and features like node and edge transparency.
The tight connection between NDEx and Cytoscape is now stronger than ever: with NDEx 2.2.1, user can choose between 2 different apps to round trip networks between NDEx and Cytoscape.
- CyNDEx v4.0.3
This is the original NDEx Cytoscape app and can be used with Cytoscape v3.3 or higher. There are no new features in this version of the CynDEx app; click the version number to access the app's store page and review the details about bug fixes and other improvements.
If you need help with the CyNDEx app, please review our updated CyNDEx Tutorial.
- CyNDEx-2 v2.2.1
CyNDEx-2 is the latest NDEx Cytoscape app and can only be used with Cytoscape v3.6 or higher. CyNDEx-2 relies on the REACT framework, uses JX Browser and will become a "core app" in the upcoming Cytoscape v3.6.1 release.
CyNDEx-2 has an improved user interface that makes it easier to browse, import, annotate and export networks. In addition, it makes possible to use the new "open in Cytoscape" feature described above in this document.
If you work with Cytoscape v3.6 or higher, we recommend using CyNDEx-2 to take full advantage of allsnew features and improvements.
For instructions about the app and its use, click the version number above to access the app's store page.
NOTE: all Cytoscape collections are displayed in the NDEx UI with the a black Cytoscape icon. Any single-subnetwork
collection can be converted to single network simply by importing it into Cytoscape with CyNDEx-2 and re-exporting it to
NDEx as network; we also recommend you uncheck the option to "Save as New Network" in order to update the existing copy of the network.
NDEx 2.2.1 brings a number of important back-end changes that are documented in our Technical Literature. Please review the Network Data Model document for updates in the CX specification and server changes.
New features and other improvements in NDEx 2.2.1 also required changes in the REST API; please review carefully the NDEx API Documentation prior to using it for your work.
NDEx 2.2 Release Notes - Q3-2017 Update (v2.2)
October 17th, 2017
The NDEx Q3-2017 Update adds a number of new features to support the NDEx Project's mission to enhance collaboration while facilitatating publication and dissemination of biological networks. All new features and improvements are deployed to the NDEx Public Server as well as available in the NDEx 2.2 Installation Bundle that can be downloaded from our FTP Server.
This section provides information about the new features introduced with the Q3-2017 Update. Please review this document carefully prior to using NDEx 2.2 for your work.
- OAUTH2 Support - Beginning with v2.2, NDEx provides support for the Open Authentication protocol and allows users to log-in to their accounts using their existing Google credentials. Users can also change the email address associated to their existing NDEx account and substitute it with their Gmail address in order to take full advantage of the new feature. New users are strongly recommended to sign up using a Gmail address if they have one.
OAUTH2 support also makes it easier to connect NDEx with external applications and enable seamless workflows using a number of analytic tools already available to the scientific community.
*** Important Note: OAUTH2 support is not available when using the SAFARI web browser. ***
- Enhanced Publishing Support - In order to become an essential tool for scientific publishing, NDEx 2.2 provides new features that support authors, editors and reviewers in streamlining the entire publication process. Detailed information about Enahnced Publishing support can be found in the new user manual Publishing in NDEx.
- Licensing options: users can now choose and apply to their networks one of several standard licenses or specify a custom modified license. All available licenses are included in a dropdown menu in the Network Property Editor and embedded documentation provides access to additional information for every type of license.
- Request a DOI: NDEx 2.2 allows users to submit a request to obtain a Digital Object identifier (DOI) for their networks. DOIs are widely used in the publishing industry to provide stable, persistent access to resources (such as articles) and guarantee they remain unchanged and available to the public for an extended period of time. With the option to request DOIs, users can guarantee stable access to their biological network data in NDEx and enable its referencing in publications, grant applications or other online resources to share with the entire scientific community.
- Paging - the NDEx Web UI now uses a paging system when displaying a large number of networks. The new paging system makes the user interface faster and improves the end user experience. The paging system has a number of "smart" features such as remembering selected items when flipping through different pages to perform bulk actions.
- Notification Badges - the NDEx Web UI now displays badges in the "Tasks & Notifications" tab to visually alert users about the presence of new items requiring action. Badges will be extended in future releases to cover additional task types such as new networks upload, direct messaging, and more. At this time, badges are limited to the "Tasks & Notifications" tab and apply to:
- Exporting networks
- Accessing networks by users
- Accessing networks by groups
- Joining a group
- Revised Network Indexing Policy - beginning with NDEx v2.2, by default all networks loaded to NDEx (regardless of their privacy) will be NOT SEARCHABLE, which means they will not be returned as results in a search. This new behavior is required to optimize both server performance and the relevance of search results.
- If users want to make their networks SEARCHABLE in NDEx v2.2, they need to modify the network privacy to "PUBLIC" or "PRIVATE" using the NDEx Web UI controls.
- Searchable networks are identified by the presence of a "magnifier glass" icon in the Visibility column of the user's "My Networks" tab.
- The new indexing policy applies only to networks loaded to NDEx after the v2.2 release: all pre-existing networks will be unaffected and remain SEARCHABLE unless users specifically decide to make them "not searchable".
- Improved Provenance History - in order to optimize the size footprint of CX files for network models that are updated frequently (such as "The RAS Machine"), the default NDEx server behavior is now set to no longer record old/new values when performing updates on a network profile; however, the update event itself is still recorded in the provenance tree.
NDEx 2.2 Installation Bundle
The new NDEx 2.2 installation bundle is now available for DOWNLOAD from our FTP Server. The bundle includes:
- NDEx 2.2 server
- NDEx 2.2 WebApp
- Combined Query Service
- The NDEx Sync 2.2 (copier program) was updated to be fully compatible with the new NDEx 2.2 server and can also be downloaded from our FTP Server.
Updated, step-by-step Installation Instructions can be found on this documentation website and on our FTP server. Please review the NDEx installation instructions carefully even if you have already successfully performed a de novo NDEx installation in the past.
NDEx Python Client
- NDEx2 v1.0 is a new Python based data model and client used for building, editing, downloading and uploading networks in NDEx. Emphasis has been placed on importing and exporting in standard formats such as Pandas Data Frames and Networkx. NDEx2 requires Python 3 and can be installed from the Python Package Index (PyPI) using PIP and the following command:
pip install ndex2. Please refer to the NDEx2 Client GitHub repository for additional info, examples and a detailed overview of all supported API functions.
- The legacy NDEx Python Client has been updated to v3.1 and, although still supported, it has been deprecated and no additional development is planned. NDEx v3.1 requires Python 2.7.9 and provides Python 3 compatibility, however if you work in Python 3, we strongly recommend you migrate to the new NDEx2 v1.0 client as soon as possible. Please refer to the NDEx Python Client v3.1 GitHub repository for additional information.
NDEx 2.1 Release Notes - Q2-2017 Update (v2.1)
June 30th, 2017
The NDEx Q2-2017 Update adds new functionalities focused on the NDEx Project's mission to enhance collaboration while facilitatating publication and dissemination of biological networks. All new features and improvements are deployed to the NDEx Public Server as well as available in the NDEx 2.1 Installation Bundle that can be downloaded from our FTP Server.
This section lists all the new features introduced with this Q2-2017 Update: some of the features have limitations that will be addressed in future releases as we gather more feedback and usage statistics.
*** Note to developers: please review the NDEx API Documentation for more information about new functions related to the New Features listed below. ***
- Search Term Expansion: when searching for genes and/or proteins, users can now decide to expand their search terms to all possible synonyms/aliases available. These feature makes searching networks easier and more efficient as networks in NDEx can be annotated using a number of different identifiers and nomenclature standards.
Note: Search Term Expansion is only available for genes and proteins and is not compatible with the use of boolean operators or other Lucene Syntax.
Please review the manual about Finding and Querying Networks for detailed information and examples about searching networks in NDEx.
- Network Sets: users can now create and share Network Sets, "folders" where networks can be grouped and organized in collections.
- Each Network Set can be assigned a specific name, description and reference in case its network content has been already published in a scientific journal.
- Networks can be included in a Set regardless of their ownership and visibility, which means that "user A" can create a set that includes his own networks as well as networks owned by other NDEx users as long as he can access them.
- Networks can easily be added or removed from a set at any time.
- Sets can be "showcased" to control their visibility on the owner's page.
- Sets can be "shared" using the new "Share URL" feature described below.
- As of NDEx 2.1, Network Sets are not returned in the search results: so, in order to see a Network Set, users need to visit the Set's owner page.
- Share URL: individual networks as well as Network Sets can now be shared easily with anyone. This feature works like the sharable URL in Google Docs or Dropbox: users can enable the Share URL for a certain network (or Set) and send it to any recipients by pasting in the body of an email or including it in a paper submission or grant proposal.
- The Share URL feature is only available for PRIVATE networks; public networks are visible to anyone and can easily be shared by copying the URL displayed in the browser's address bar.
- Users can disable the Share URL feature at any time if they wish to restrict access to a particular network (or Set).
- Disk Space Quota: every user in NDEx has a generous 9.3 GB disk space allowance to store networks. The amount of disk spaced used is shown on the left hand side in the user's account page. If your account needs additional disk space, please contact us.
- Clone: users can now create a personal copy of any NDEx network using the new Clone function. This feature is useful when users want to modify/update a network but cannot operate on the original one because it is a public reference network and therefore not editable.
- Any networks can be cloned regardless of their visibility as long has you have access to them.
- Cloning a network will affect your disk space allowance.
- Cloning events are recorded in the network provenance history.
- Consolidated Network Property Editor: all network properties, including "Visibility", are now grouped in the new Network Property Editor that can be accessed using the "Edit Properties" button in the network view page.
The Network Property Editor also displays an "NDEx Score" for the network, that depends on its degree of annotation.
- NDEx Score: the network score is a very important parameter as it determines how networks rank in the search results: if a network is very well annotated (i.e. NDEx Score = 80), it will be returned at the top of the search results while if it is poorly annotated (i.e NDEx Score = 10) it will be pushed down to the bottom.
- The NDEx Score can range anywhere from "0" (Critical) to "115" (Top).
- The more network properties are defined, the higher the score.
- Custom properties can be added but their contribution to the overall NDEx Score is limited.
NDEx 2.1 Installation Bundle
The new NDEx 2.1 installation bundle is now available for DOWNLOAD from our FTP Server. The bundle includes:
- NDEx 2.1 server
- NDEx 2.1 WebApp
- Combined Query Service
- NDEx Sync 2.0 (copier program)
Updated, detailed Installation Instructions can be found on this documentation website. Please review the NDEx v2.1 installation instructions carefully even if you have already successfully performed a de novo> NDEx installation in the past.
NDEx R Client
The NDEx R Client (ndexR) is a module that simplifies access to the NDEx Server API and provides convenience methods for common operations on networks. It was developed by the Kramer Lab at the Univeristy of Goettingen, Germany and is available as a Bioconductor Package. Please review the NDEx R Client doc for additional info and a useful "ndexR cheat sheet"!
The Q2-2017 Update also provides access to updated pathways and networks: these include but are not limited to:
In addititon, new data driven networks are being generated and progressively loaded to NDEx:
All new and updated networks provide direct links to relevant external databases and include custom visual styles and layouts.
NDEx 2.0 Release Notes - Q1-2017 Update (v2.0.1)
April 7th, 2017
Besides bug fixes and general improvements, the Q1-2017 Update restores a numberof functionalities that had been temporarily disabled in the NDEx 2.0 official release back in December 2016. In addition, the NDEx 2.0 installation bundle is now available for download from our FTP server.
Web User Interface
These are the NDEx web UI changes introduced in the Q1-2017 Update:
- Re-enabled and expanded "Network Property" editor: network properties for individual networks and Cytoscape collections with 1 subnetwork can now be edited again in the NDEx UI. Editing network properties for Cytoscape collections with more than 1 subnetwork is not supported in the NDEx UI: such collections should be edited directly in Cytoscape. In addition, the schema of available network properties that are indexed by our Solr server has been expanded. An updated table showing all indexed properties can be found in the user manual Finding and Querying Networks.
- Re-enabled "Bulk Actions": Change description, Change reference and Change version can now be used again to set these properties on multiple networks simultaneously. As for the network properties mentioned above, bulk actions are not available to operate on Cytoscape collections that have more than 1 subnetwork.
- Added logo and warnings for Cytoscape collections: the UI now displays a black Cytoscape icon in the network table to easily identify Cytoscape collections that have more than 1 subnetwork. Mouse-over or clicking the icon displays a warning message.
- Re-enabled the "Advanced Query" feature: networks can now be queried not only by keywords but also by properties associated with their nodes and/or edges. This feature is handled by our new stand-alone combined query engine using the latest version of the NDEx Python library. The new service is plugged into the existing user interface and no major change will be experienced end users. !!! Important Note !!!: the combined query service is not available for Cytoscape collections with more than 1 subnetwork.
Supported Network File Formats
The Q1-2017 Update introduces the "GSEA Gen Set" export format; this feature generates a text file (.grp) containing unique human gene symbols that can be used for enrichment analysis. Additional export formats will be added in future updates.
NDEx 2.0 Installation Bundle
The new NDEx 2.0 installation bundle is now available for DOWNLOAD from our FTP Server. The bundle includes:
- NDEx 2.0 server
- NDEx 2.0 WebApp
- Combined Query Service
- NDEx Sync 2.0 (copier program)
Updated, detailed Installation Instructions can be found on this documentation website.
- The copier program (NDEx Sync 2.0) is not back compatible with prior NDEx releases and can only synchronize/update content between servers running NDEx 2.0. For more information, please review the technical document about Using the NDEx Sync Program.
- Operations on Cytoscape collections with more than 1 subnetwork are limited to uploading (via CyNDEx), sharing and downloading.
NDEx 2.0 Release Notes
December 21st, 2016
The NDEx v2.0 release provides a major improvement in performance, scalability and reliability. Our new, streamlined NDEx 2.0 server and API bring you faster upload/download speeds to handle larger networks, support a larger user-base and heavier traffic loads. Features such as our “Neighborhood Query” have been decoupled from the core server and deployed as stand alone services thus creating a more flexible infrastructure. Improvements in the web UI simplify the end user experience and full support for CyNDEx 3.3.0 strengthen the already tight bond between NDEx and Cytoscape.
Some features of NDEx have been temporarily deactivated in v2.0; we will be adding back these features following a modular, short-cycle release strategy. A few features have been permanently removed for clarity, stability or simply lack of use. Please take a moment to carefully review this document before using NDEx 2.0.
!!! NDEx 2.0 is currently available ONLY on the NDEx Public server: an installation bundle, relative installation instructions and the new copier program (ndex-sync 2.0) will be released later in Q1 2017 !!!
Thanks for using NDEx and please contact us with bugs, suggestions, and requests. We would especially like to hear about the ways in which you would like to use biological networks, either as an end user or application developer: this will help us plan the next phases of NDEx development.
The NDEx Team
Web User Interface
These are some of the web UI changes we have introduced in NDEx 2.0:
Redesign of main NDEx landing page: new “Featured Collections” help you get started using NDEx.
Improved NDEx network viewer: result of a neighborhood query now retain the graphic style of the parent network.
Improved search results table: we added a quick download icon to easily retrieve any network in CX format; we also added a Reference column, Status column as well as Disease and Tissue. The new “Showcase” feature allows you to control which networks are publicly displayed in your user account page. Finally, we removed the Format column.
Added e-mail verification: in NDEx 2.0, you need to verify your e-mail address when creating a new account. After signing up, the UI enables you to view and update your email address.
Removed “Archive” and “Download” features for BEL Namespace files: this feature will be replaced as part of the upcoming release of the new BEL Uploader
Temporarily disabled the “Network Property” editor: although NDEx 2.0 allows Cytoscape users to save and access “collections” containing multiple sub-networks, it currently cannot handle editing of network properties for individual sub-networks. Therefore, we have temporarily disabled the “Network Property” editor for Cytoscape collections that have more than 1 subnetwork.
Added “Bulk Action”:the “read-only” flag can now be set/unset for multiple networks at the same time.
Disabled some “Bulk Actions”: Change description, change reference and change version are temporarily unavailable for any networks.
Improved “Neighborhood Query” feature: this feature is now deployed as a stand alone service using the latest version of the NDEx Python client. Nothing changed in the user interface. More info about this feature are available below.
Temporarily disabled the “Advanced Query” feature: the advanced query feature is currently in development and will be restored in Q1 2017.
Supported Network File Formats
In NDEx 2.0, the web UI only allows the direct UPLOAD of networks in CX (.cx) format.
!!! IMPORTANT NOTICE !!!
Other network formats, including but not limited to BioPAX Level 3 (.owl), XGMML (.xgmml) and SIF/Extended Binary SIF (.sif), can be loaded to NDEx directly from Cytoscape using the CyNDEx App available in the Cytoscape App Store. Please refer to the CyNDEx tutorial for additional info.
NDEx 2.0 allows to DOWNLOAD networks only as CX (.cx) files through the new “Quick Download” icon in the NDEx UI.
Additional planned EXPORT formats include SIF, Microsoft Excel, TSV, GSEA and will be added later in Q1 2017.
The NDEx 2.0 Network Query is a fast, convenient way to view a subset of your network. Query performance is increased by 30% for networks with up to 10,000 edges; in addition, the Query service is now deployed as a stand-alone component and completely decoupled from the core NDEx server, thus allowing the release of improvements and new features independently of server updates. No changes were made in the web UI Query controls.
NDEx 2.0 API
For developers, the REST API has been streamlined and clarified. Please review the new NDEx 2.0 API documentation for details.
Full CyNDEx support
Cytoscape and NDEx are the two main components of the new service-oriented “Cytoscape Cyberinfrastructure”. NDEx 2.0 therefore fully supports the previously released Cytoscape API. You can keep using CyNDEx v3.3.0 with your favorite Cytoscape version (3.3 or higher) and save your networks to either an NDEx v1.3 server or the new NDEx 2.0 Public server.
Supported Web Browsers
We recommend that you always update your preferred web browser to the latest version whenever possible. NDEx fully supports the following browsers:
– Google Chrome
– Mozilla Firefox
The copier program (NDEx Sync v1.3.3) is not compatible with the new NDEx 2.0. We will release a new copier program later in Q1 2017.