ISMB 2014: BioinfoCoreWorkshopWriteUp - Revision history

Hrh at 15:29, 14 November 2014

2014-11-14T15:29:28Z

Alastair.kerr: /* Topic 1: Core on a Budget vs Enterprise */

2014-07-22T15:37:00Z

Topic 1: Core on a Budget vs Enterprise

Alastair.kerr: /* Topic 1: Core on a Budget vs Enterprise */

2014-07-22T15:32:54Z

Topic 1: Core on a Budget vs Enterprise

Alastair.kerr: /* Discussion */

2014-07-22T11:52:43Z

Discussion

Alastair.kerr: /* Topic 1: Core on a Budget vs Enterprise */

2014-07-22T09:34:08Z

Topic 1: Core on a Budget vs Enterprise

Simon andrews: /* Topic 1: Core on a Budget vs Enterprise */

2014-07-22T08:01:14Z

Topic 1: Core on a Budget vs Enterprise

Simon andrews: /* Topic 1: Core on a Budget vs Enterprise */

2014-07-15T15:57:55Z

Topic 1: Core on a Budget vs Enterprise

Simon andrews: /* Discussion */

2014-07-15T15:47:45Z

Discussion

Simon andrews: /* Discussion */

2014-07-15T15:47:28Z

Discussion

Simon andrews: /* Discussion */

2014-07-15T15:47:04Z

Discussion

← Older revision		Revision as of 15:29, 14 November 2014
Line 77:		Line 77:

	=== Funding your core ===		=== Funding your core ===
		+
		+
		+
		+	== Followup: InfrastructureForNewCores ==
		+
		+	see: [[ISMB_2014:_InfrastructureForNewCores]]

@@ Line 22: / Line 22: @@
 Alastair Kerr led the session talking about his [http://bifx1.bio.ed.ac.uk/ small core (2 people) at Edinburgh University].  His core is almost completely self-reliant and has to cover all of the hardware, storage, software and analysis infrastructure required for the full range of users he supports.  Alastair described how his infrastructure is built on a number of key open source components from [http://en.wikipedia.org/wiki/ZFS ZFS] based storage systems which provide 0.5PB of storage for a fraction of the cost of commercial systems to pipelining and workflow systems built within [http://galaxyproject.org/ Galaxy], to user-friendly analysis scripts provided to users though the [http://shiny.rstudio.com/ R Shiny system].
-Alastair described how he actively avoids the use of commercial software within his group and described occasions in the past where their adoption of initially useful commercial packages had ultimately had negative impacts when the software later changed its licensing fees or became unsupported.  The only commercial package they still have is [http://www.dnastar.com/t-allproducts.aspx Lasergene] for basic molecular biology manipulations and this is mostly for historical reasons and for the lack of a suitable open alternative. A key benefit with this choice of open source infrastructure is that the process of data analysis from raw data to paper figure can be shared and used by anyone. Alastair's group facilitates this process by ensuring that any scripts developed in-house are shared in either [https://toolshed.g2.bx.psu.edu/ Galaxy Toolshed] or [http://github.com github].
+Alastair described how he actively avoids the use of commercial software within his group and described occasions in the past where their adoption of initially useful commercial packages had ultimately had negative impacts when the software later changed its licensing fees or became unsupported.  The only commercial package they still have is [http://www.dnastar.com/t-allproducts.aspx Lasergene] for basic molecular biology manipulations and this is mostly for historical reasons and for the lack of a suitable open alternative.
 Mike Poidinger then went on to present the contrary case.  His group is very well funded by his [http://www.a-star.edu.sg/ supporting institution] and is somewhat larger than the Edinburgh group with 9 members.  Mike's initial contention was that it should be a requirement when setting up a core that sufficient funding be provided and that it would be reasonable to refuse to head up a core where suitable funding to provide an appropriate infrastructure was not forthcoming.  Mike stressed that open source software played a major role in the operation of his core, with much of the analysis of data being provided by these types of packages, which are generally much more agile and able that their commercial equivalents.  However, he made a strong case for two particular pieces of commercial support software which now form a key part of his infrastructure - [http://accelrys.com/products/pipeline-pilot/ Pipeline Pilot] and [http://spotfire.tibco.com/ Spotfire].

@@ Line 22: / Line 22: @@
 Alastair Kerr led the session talking about his [http://bifx1.bio.ed.ac.uk/ small core (2 people) at Edinburgh University].  His core is almost completely self-reliant and has to cover all of the hardware, storage, software and analysis infrastructure required for the full range of users he supports.  Alastair described how his infrastructure is built on a number of key open source components from [http://en.wikipedia.org/wiki/ZFS ZFS] based storage systems which provide 0.5PB of storage for a fraction of the cost of commercial systems to pipelining and workflow systems built within [http://galaxyproject.org/ Galaxy], to user-friendly analysis scripts provided to users though the [http://shiny.rstudio.com/ R Shiny system].
-Alastair described how he actively avoids the use of commercial software within his group and described occasions in the past where their adoption of initially useful commercial packages had ultimately had negative impacts when the software later changed its licensing fees or became unsupported.  The only commercial package they still have is [http://www.dnastar.com/t-allproducts.aspx Lasergene] for basic molecular biology manipulations and this is mostly for historical reasons and for the lack of a suitable open alternative.
+Alastair described how he actively avoids the use of commercial software within his group and described occasions in the past where their adoption of initially useful commercial packages had ultimately had negative impacts when the software later changed its licensing fees or became unsupported.  The only commercial package they still have is [http://www.dnastar.com/t-allproducts.aspx Lasergene] for basic molecular biology manipulations and this is mostly for historical reasons and for the lack of a suitable open alternative. A key benefit with this choice of open source infrastructure is that the process of data analysis from raw data to paper figure can be shared and used by anyone. Alastair's group facilitates this process by ensuring that any scripts developed in-house are shared in either [https://toolshed.g2.bx.psu.edu/ Galaxy Toolshed] or [http://github.com github].
 Mike Poidinger then went on to present the contrary case.  His group is very well funded by his [http://www.a-star.edu.sg/ supporting institution] and is somewhat larger than the Edinburgh group with 9 members.  Mike's initial contention was that it should be a requirement when setting up a core that sufficient funding be provided and that it would be reasonable to refuse to head up a core where suitable funding to provide an appropriate infrastructure was not forthcoming.  Mike stressed that open source software played a major role in the operation of his core, with much of the analysis of data being provided by these types of packages, which are generally much more agile and able that their commercial equivalents.  However, he made a strong case for two particular pieces of commercial support software which now form a key part of his infrastructure - [http://accelrys.com/products/pipeline-pilot/ Pipeline Pilot] and [http://spotfire.tibco.com/ Spotfire].

@@ Line 39: / Line 39: @@
 Much of the subsequent discussion for the session focussed around whether there were individual or groups of commercial packages for which there wasn't a suitable free and open alternative.  The major area which came up was for packages providing functional annotation, with the main contenders being [http://www.ingenuity.com/products/ipa Ingenuity IPA] and [http://lsresearch.thomsonreuters.com/ | GeneGo Metcore].  Several sites are paying for these types of packages and the consensus was that what you're paying for wasn't the software but rather the expanded set of gene lists and connections which have been manually mined from the literature by these companies.
-These types of system are generally liked by users as they provide an easy way into the biology of an analysis result.  They offer some advantages over equivalent open source products, but their major open competitors such as [http://david.abcc.ncifcrf.gov/ DAVID] and [http://www.genemania.org/ GeneMania] are also very good and well used.
+These types of system are generally liked by users as they provide an easy way into the biology of an analysis result.  They offer some advantages over equivalent open source products, but their major open competitors such as [http://david.abcc.ncifcrf.gov/ DAVID], [http://cbl-gorilla.cs.technion.ac.il/ GOrilla] and [http://www.genemania.org/ GeneMania] are also very good and well used.
 There was a general opinion that the costs and licensing terms for the commercial annotation packages were quite severe.  This was especially the case for IPA where some sites had starting to do cost recovery for the licence and found that many of the previous users weren't prepared to pay the costs for this.  MetaCore licensing was more flexible with the ability to buy licences for a given number of simultaneous users which fitted better for many people's use cases.

@@ Line 15: / Line 15: @@
 Speakers:
-* Alastair Kerr - Edinburgh Uni
+* Alastair Kerr - Edinburgh Uni [[media:AK-ismb-coreshop.pdf|Slides]]
 * Mike Poidinger - A*Star [[media:Mpoidinger_ismb2014.pdf|Slides]]

@@ Line 20: / Line 20: @@
 The purpose of this session was to look at whether it is possible to run a core facility on a limited budget, and to explore what becomes possible when you have a larger amount of money to spend.
-Alastair Kerr led the session talking about his small core (2 people) at Edinburgh University.  His core is almost completely self-reliant and has to cover all of the hardware, storage, software and analysis infrastructure required for the full range of users he supports.  Alastair described how his infrastructure is built on a number of key open source components from ZFS based storage systems which provide 0.5PB of storage for a fraction of the cost of commercial systems to pipelining and workflow systems built within Galaxy, to user-friendly analysis scripts provided to users though the R Shiny system.
+Alastair Kerr led the session talking about his [http://bifx1.bio.ed.ac.uk/ small core (2 people) at Edinburgh University].  His core is almost completely self-reliant and has to cover all of the hardware, storage, software and analysis infrastructure required for the full range of users he supports.  Alastair described how his infrastructure is built on a number of key open source components from [http://en.wikipedia.org/wiki/ZFS ZFS] based storage systems which provide 0.5PB of storage for a fraction of the cost of commercial systems to pipelining and workflow systems built within [http://galaxyproject.org/ Galaxy], to user-friendly analysis scripts provided to users though the [http://shiny.rstudio.com/ R Shiny system].
-Alastair described how he actively avoids the use of commercial software within his group and described occasions in the past where their adoption of initially useful commercial packages had ultimately had negative impacts when the software later changed its licensing fees or became unsupported.  The only commercial package they still have is Lasergene for basic molecular biology manipulations and this is mostly for historical reasons and for the lack of a suitable open alternative.
+Alastair described how he actively avoids the use of commercial software within his group and described occasions in the past where their adoption of initially useful commercial packages had ultimately had negative impacts when the software later changed its licensing fees or became unsupported.  The only commercial package they still have is [http://www.dnastar.com/t-allproducts.aspx Lasergene] for basic molecular biology manipulations and this is mostly for historical reasons and for the lack of a suitable open alternative.
-Mike Poidinger then went on to present the contrary case.  His group is very well funded by his supporting institution and is somewhat larger than the Edinburgh group with 9 members.  Mike's initial contention was that it should be a requirement when setting up a core that sufficient funding be provided and that it would be reasonable to refuse to head up a core where suitable funding to provide an appropriate infrastructure was not forthcoming.  Mike stressed that open source software played a major role in the operation of his core, with much of the analysis of data being provided by these types of packages, which are generally much more agile and able that their commercial equivalents.  However, he made a strong case for two particular pieces of commercial support software which now form a key part of his infrastructure - Pipeline Pilot and Spotfire.
+Mike Poidinger then went on to present the contrary case.  His group is very well funded by his [http://www.a-star.edu.sg/ supporting institution] and is somewhat larger than the Edinburgh group with 9 members.  Mike's initial contention was that it should be a requirement when setting up a core that sufficient funding be provided and that it would be reasonable to refuse to head up a core where suitable funding to provide an appropriate infrastructure was not forthcoming.  Mike stressed that open source software played a major role in the operation of his core, with much of the analysis of data being provided by these types of packages, which are generally much more agile and able that their commercial equivalents.  However, he made a strong case for two particular pieces of commercial support software which now form a key part of his infrastructure - [http://accelrys.com/products/pipeline-pilot/ Pipeline Pilot] and [http://spotfire.tibco.com/ Spotfire].
-Mike's contention was that whilst open source packages are very good at performing individual analyses, they can be difficult to work with due to the difficulty in collating and understanding the wide variety of output files and formats they generate.  His group uses pipeline pilot to abstract away a lot of the 'dirty' parts of an analysis so that they can leave the commercial system to store and retrieve appropriate data and to handle the format conversions required to pass data through several parts of a larger pipeline.  Having this type of collation system in place means that all of the analysis can be done in the form of pipelines and a complete record of all analyses is preserved and can be reproduced or reused very easily.
+Mike's contention was that whilst open source packages are very good at performing individual analyses, they can be difficult to work with due to the difficulty in collating and understanding the wide variety of output files and formats they generate.  His group uses [http://accelrys.com/products/pipeline-pilot/ pipeline pilot] to abstract away a lot of the 'dirty' parts of an analysis so that they can leave the commercial system to store and retrieve appropriate data and to handle the format conversions required to pass data through several parts of a larger pipeline.  Having this type of collation system in place means that all of the analysis can be done in the form of pipelines and a complete record of all analyses is preserved and can be reproduced or reused very easily.
-The other package heavily used within his group is Spotfire.  This is a data presentation and integration package which makes it easy for users to explore the quantitative data coming out of the end of an analysis pipeline.  It would compete with simple solutions such as Excel, or more complex analyses and visualisations in R, but provides a friendly and powerful interface to the data.  Mike's team have linked these packages to other tools such as the Moin moin wiki to provide a combined system which keeps a complete record of analyses, presents it back to the original users in a friendly way and provides an interface through which they can themselves manipulate and explore the data further.
+The other package heavily used within his group is [http://spotfire.tibco.com/ Spotfire].  This is a data presentation and integration package which makes it easy for users to explore the quantitative data coming out of the end of an analysis pipeline.  It would compete with simple solutions such as Excel, or more complex analyses and visualisations in R, but provides a friendly and powerful interface to the data.  Mike's team have linked these packages to other tools such as the Moin moin wiki to provide a combined system which keeps a complete record of analyses, presents it back to the original users in a friendly way and provides an interface through which they can themselves manipulate and explore the data further.
 Overall it was Mike's contention that the use of these commercial products within his group added around 20% to the efficiency of his staff, and also allowed new members to get started much more quickly.  The cost of the licensing for these packages was therefore outweighed by the efficiency improvements which his group gained from their use.

@@ Line 35: / Line 35: @@
 There were some questions about the talks which had been presented.  There was some lively discussion about the financial benefits of using commercial software, with some people arguing that the amount of money spent on a big commercial system would fund an additional FTE and that this would be a more productive use of the funding.  Whilst a consensus was not really reached on this point, it seems that the merits of this, and possibly several other commercial / open decisions depends on the scale of your group.  Smaller cores are more able to support their own infrastructure but as the size of the group or the community expands then the support of infrastructure becomes more of a burden.  At this point getting commercial support for storage, pipelining or data management becomes more attractive and allows the core to focus on the science rather than the specifics of the platforms being used.
-A suggestion which came out of this discussion was that bioinfo-core could try to collate some ideas about what infrastructure would be useful to put in place when establishing a new core.  The idea would be that we could generate a basic check list of the types of components you would want and give some options for available solutions for each area and add comments about the merits of each.  To this end we've set up a [[ISMB_2014:_InfrastructureForNewCores basic template page]] which we can expand after further discussion on the list.
+A suggestion which came out of this discussion was that bioinfo-core could try to collate some ideas about what infrastructure would be useful to put in place when establishing a new core.  The idea would be that we could generate a basic check list of the types of components you would want and give some options for available solutions for each area and add comments about the merits of each.  To this end we've set up a [[ISMB_2014:_InfrastructureForNewCores | basic template page]] which we can expand after further discussion on the list.
 Much of the subsequent discussion for the session focussed around whether there were individual or groups of commercial packages for which there wasn't a suitable free and open alternative.  The major area which came up was for packages providing functional annotation, with the main contenders being [http://www.ingenuity.com/products/ipa Ingenuity IPA] and [http://lsresearch.thomsonreuters.com/ | GeneGo Metcore].  Several sites are paying for these types of packages and the consensus was that what you're paying for wasn't the software but rather the expanded set of gene lists and connections which have been manually mined from the literature by these companies.

@@ Line 33: / Line 33: @@
 === Discussion ===
-Much of the initial discussion for the session focussed around whether there were individual or groups of commercial packages for which there wasn't a suitable free and open alternative.  The major area which came up was for packages providing functional annotation, with the main contenders being [http://www.ingenuity.com/products/ipa Ingenuity IPA] and [http://lsresearch.thomsonreuters.com/ | GeneGo Metcore].  Several sites are paying for these types of packages and the consensus was that what you're paying for wasn't the software but rather the expanded set of gene lists and connections which have been manually mined from the literature by these companies.
+There were some questions about the talks which had been presented.  There was some lively discussion about the financial benefits of using commercial software, with some people arguing that the amount of money spent on a big commercial system would fund an additional FTE and that this would be a more productive use of the funding.  Whilst a consensus was not really reached on this point, it seems that the merits of this, and possibly several other commercial / open decisions depends on the scale of your group.  Smaller cores are more able to support their own infrastructure but as the size of the group or the community expands then the support of infrastructure becomes more of a burden.  At this point getting commercial support for storage, pipelining or data management becomes more attractive and allows the core to focus on the science rather than the specifics of the platforms being used.
 These types of system are generally liked by users as they provide an easy way into the biology of an analysis result.  They offer some advantages over equivalent open source products, but their major open competitors such as [http://david.abcc.ncifcrf.gov/ DAVID] and [http://www.genemania.org/ GeneMania] are also very good and well used.