### DEPLOYMENT STATUS Everyone, please add a note here each time you log in and do ANYTHING to the production deployment. Specify which version (tag) is currently deployed. For a description of features in each BRANCH, log in to ```portal2test@8ball.sdsc.edu``` and see ```~/cipres-portal-src/Readme.txt```. REMEMBER to get rid of ```tool-registry.cfg.xml.new``` when it's no longer needed or ```tl.sh``` will keep using it. --- Wed Jul 6 08:38:45 PDT 2016 TODO: NEXT BRANCH, r38 (i.e. when current trunk goes into production) requires JAVA 8 !!! Also requires that 'sdrun org.ngbw.utils.CreateHomeFolders' be built and run before the app is brought up. Already made this changes to build.properties. This will be needed with branch r38. If there's a problem, can go back to build.properties.save and branch r37. - change triton.keyFile to just keyFile (or add keyFile and keep triton.keyFile, in case we have to rollback to earlier branch). - add ```teragrid.comet.runner=org.ngbw.sdk.common.util.GsiSSHProcessRunner``` ```teragrid.gordon.runner=org.ngbw.sdk.common.util.GsiSSHProcessRunner``` --- Tue Jun 08 09:00:00 PST 2021 Tony: portal-R43.111. * Patches provided by Mark M. Fix retrieving country code error. --- Wed Jun 02 14:42:00 PST 2021 Tony: portal-R43.110. * Force users to update their passwords if the password contains one or more unsupported chars. --- Tue May 25 13:45:00 PST 2021 Tony: portal-R43.109. * Patches provided by Mark M. --- Wed May 20 16:02:00 PST 2021 mzhuang: portal-R43.108. * Patches provided by Mark M. --- Wed May 05 08:35:00 PST 2021 Tony: portal-R43.107. * Patches provided by Mark M. --- Thu Apr 22 13:00:00 PST 2021 Tony: portal-R43.106. * Patches provided by Mark M. --- Fri Apr 16 23:35:00 PST 2021 Tony: portal-R43.105. * Patches provided by Mark M. --- Wed Apr 07 19:40:00 PST 2021 Tony: portal-R43.104. * Patches provided by Mark M. Fix registration error. --- Tue Mar 30 09:50:00 PST 2021 Tony: portal-R43.103. * Patches provided by Mark M. --- Sun Mar 28 00:15:00 PST 2021 Tony: portal-R43.102. * Fix missing Phoenix subscription link. --- Wed Mar 24 16:25:00 PST 2021 Tony: portal-R43.101. * Patches provided by Mark M. --- Tue Mar 23 09:40:00 PST 2021 Tony: portal-R43.100. * Patches provided by Mark M. --- Fri Mar 19 09:25:00 PST 2021 Tony: portal-R43.99. * Patches provided by Mark M. --- Thu Mar 18 09:25:00 PST 2021 Tony: portal-R43.98. * Patches provided by Mark M. --- Sat Mar 13 10:00:00 PST 2021 Tony: portal-R43.97. * Patches provided by Mark M. --- Thu Mar 11 16:45:00 PST 2021 Tony: portal-R43.96. * Patches provided by Mark M. --- Fri Mar 05 09:15:00 PST 2021 Tony: portal-R43.95. * Patches provided by Mark M. --- Thu Feb 25 17:00:00 PST 2021 Tony: portal-R43.94. * Patches provided by Mark M. --- Thu Feb 25 10:40:00 PST 2021 Tony: portal-R43.93. * Patches provided by Mark M. --- Tue Feb 23 16:50:00 PST 2021 Tony: portal-R43.92. * Patches provided by Mark M. --- Tue Feb 23 08:50:00 PST 2021 Tony: portal-R43.91. * Add a checker to filter out unsupported characters in user’s password. --- Thu Feb 04 09:20:00 PST 2021 Tony: portal-R43.90. * Fix a syntax error in SQL statement in Statistics class. --- Wed Feb 03 13:15:00 PST 2021 Tony: portal-R43.89. * Set SU_COMPUTED to 0 if user terminates the job before it starts running. --- Mon Feb 01 17:30:00 PST 2021 Tony: portal-R43.88. * Handle errors from Phoenix nicely and allow users to continue logging into CIPRES portal. --- Mon Feb 01 10:45:00 PST 2021 Tony: portal-R43.87. * Set ‘conversion’ to 1.00 in JobCharge.getCharge() since the computed value is for storing in the database, not for reporting to the user. --- Thu Jan 28 14:30:00 PST 2021 Tony: portal-R43.86. * Modify SU calculation module to improve efficiency. See SVN rev. 2517 for details. --- Wed Jan 27 17:19:00 PST 2021 mzhuang: portal-R43.85. * Use account TG-DEB090011 when sending jobs to expanse and also patches provided by Mark M. --- Mon Jan 20 14:00:00 PST 2021 Tony: portal-R43.84. * Output more log when creating a job and calculating predicted SU. --- Mon Jan 20 08:25:00 PST 2021 Tony: portal-R43.83. * When a CIPRES non-US user logs in, check if the user is registered with Phoenix system using one of their endpoints. * Update phoenix-sdk to version 1.2.0. --- Mon Jan 19 09:50:00 PST 2021 Tony: portal-R43.82. Modify SU calculation module to calculate predicted SU more accurately for jobs run on different resources such as Comet and Expanse. Modify XSEDE Attribute Reporting module to include jobs run on Expanse (expanse and expanse-gpu). SVN revision 2500. --- Fri Jan 15 09:30:00 PST 2021 Tony: portal-R43.81. * Patches provided by Mark M. --- Thu Jan 14 08:25:00 PST 2021 Tony: portal-R43.80. * Patches provided by Mark M. --- Fri Jan 08 12:10:00 PST 2021 Tony: portal-R43.79. * Patches provided by Mark M. --- Mon Jan 04 10:00:00 PST 2021 Tony: portal-R43.78. * Patches provided by Mark M. --- Mon Dec 21 11:20:00 PST 2020 Tony: portal-R43.77. * Add a new Workbench.hasPhoenixModuleTakenEffect() method to control more accurately on whether or not the new calculation model has been in effect. --- Fri Dec 18 14:00:00 PST 2020 Tony: portal-R43.76. * Remove internal control in XsedeAttributeReportingScheduler.save() method. --- Fri Dec 18 15:15:00 PST 2020 Tony: portal-R43.75. * Modify ApplicationPreferenceLoader class to return default value if a preference is not found in the database. * Some minor syntax modifications in Workbench, PortalStartupManager, and StringUtils classes. --- Fri Dec 18 14:16:00 PST 2020 Tony: portal-R43.74. * Patches provided by Mark M. --- Mon Dec 14 10:05:00 PST 2020 Tony: portal-R43.73. * Fix a bug in org.ngbw.sdk.Workbench.getAllocationsValidDays() method. * Add more debugging message output in XSEDE attribute reporting classes. --- Mon Dec 14 10:05:00 PST 2020 Tony: portal-R43.73. * Fix a bug in org.ngbw.sdk.Workbench.getAllocationsValidDays() method. * Add more debugging message output in XSEDE attribute reporting classes. --- Sat Dec 12 11:45:00 PST 2020 Tony: portal-R43.72. * Reimplement Phoenix related classes to avoid lib conflict with Globus GSI lib. --- Wed Dec 02 15:10:00 PST 2020 Tony: portal-R43.71. * Patches provided by Mark M. * Remove CyVerse login. --- Sat Nov 14 13:50:00 PST 2020 Tony: portal-R43.70. * Patches provided by Mark M. --- Thu Nov 12 09:05:00 PST 2020 Tony: portal-R43.69. * Patches provided by Mark M. --- Tue Nov 10 18:28:00 PST 2020 mzhuang: portal-R43.68. * Patches provided by Mark M. --- Tue Nov 10 09:05:00 PST 2020 Tony: portal-R43.67. * Patches provided by Mark M. --- Mon Nov 09 12:40:00 PST 2020 Tony: portal-R43.66. * Patches provided by Mark M. --- Mon Nov 09 11:18:00 PST 2020 mzhuang: portal-R43.65. * Patches provided by Mark M. --- Sat Nov 07 10:38:00 PST 2020 mzhuang: portal-R43.64. * For RAXML, split tasks-per-node and cups-per-task. --- Wed Nov 04 20:05:00 PST 2020 Tony: portal-R43.63. * Patches provided by Mark M. --- Tue Nov 03 11:40:00 PST 2020 Tony: portal-R43.62. * Patches provided by Mark M. --- Mon Oct 19 09:28:00 PST 2020 mzhuang: portal-R43.61. * Send jobs to multiple resources; add "expanse" resource. --- Thu Oct 15 09:55:00 PST 2020 Tony: portal-R43.60. * Force sync BEAST/BEAST2 parsers. --- Fri Sep 09 12:45:00 PST 2020 Tony: portal-R43.59. * Patches provided by Mark M. --- Tue Sep 22 16:56:00 PST 2020 mzhuang: portal-R43.58. * change to for reset mail and patches provided by Mark M. --- Mon Aug 31 10:20:00 PST 2020 Tony: portal-R43.57. * Integrate Allen Z’s design improvements for auto update parse status in dataList.jsp. --- Sat Aug 22 10:35:00 PST 2020 Tony: portal-R43.56. * Patches provided by Mark M. --- Thu Aug 20 11:15:00 PST 2020 Tony: portal-R43.55. * Allen Z fix the auto parse status update in dataList.jsp, struts.xml, and DataManager.java. --- Fri Aug 04 13:29:00 PST 2020 mzhuang: portal-R43.54. * Add "Account Activation" to Admin Interface. --- Fri Jul 31 09:05:00 PST 2020 Tony: portal-R43.53. * Patches provided by Mark M. --- Wed Jul 29 13:50:00 PST 2020 Tony: portal-R43.52. * Patches provided by Mark M. --- Wed Jul 29 09:55:00 PST 2020 Tony: portal-R43.51. * Patches provided by Mark M. --- Wed Jul 22 20:10:00 PST 2020 Tony: portal-R43.50. * Patches provided by Mark M. --- Tue Jul 14 14:45:00 PST 2020 Tony: portal-R43.49. * Patches provided by Mark M. --- Mon Jul 13 10:15:00 PST 2020 Tony: portal-R43.48. * Patches provided by Mark M. --- Tue Jul 07 21:05:00 PST 2020 Tony: portal-R43.47. * Patches provided by Mark M. --- Tue Jul 07 08:50:00 PST 2020 Tony: portal-R43.46. * Patches provided by Mark M. --- Wed Jul 01 15:30:00 PST 2020 Tony: portal-R43.45. * Fix bug preventing users from logging in with their emails. --- Wed June 24 09:25:00 PST 2020 Tony: portal-R43.44. * Usernames should not have leading and trailing spaces. Implement function to enforce it. * Resolve XSEDE Gateway Attribute Reporting library conflicts. --- Tue June 23 16:15:00 PST 2020 Tony: portal-R43.43. * Patches provided by Mark M. --- Fri June 19 13:01:00 PST 2020 Tony: portal-R43.42. * Patches provided by Mark M. --- Fri June 19 12:10:00 PST 2020 Tony: portal-R43.41. * Fix a bug where user cannot login after resetting the password. --- Thu June 18 12:15:00 PST 2020 Tony: portal-R43.40. * Fix new user's registration bug. --- Tue June 16 16:05:00 PST 2020 Tony: portal-R43.39. * Patches provided by Mark M. --- Tue June 16 15:45:00 PST 2020 Tony: portal-R43.38. * Patches provided by Mark M. **This version has a bug. Do NOT use.** --- Sat June 13 13:00:00 PST 2020 Tony: portal-R43.37. * Fix bug where users cannot change their password. --- Tue June 12 11:50:00 PST 2020 Tony: portal-R43.36. * Remove restrictions placed on CIPRES Share file transfer feature to allow all users to use it. --- Tue June 9 11:45:00 PST 2020 Tony: portal-R43.35. * Fix incorrect message when user uploads files to CIPRES Share. --- Tue June 9 11:05:00 PST 2020 Tony: portal-R43.34. * Fix SSL key exception in Globus GSISSH library. * Update message wording in email when uploading files to CIPRES Share is done. --- Fri June 5 15:00:00 PST 2020 Tony: portal-R43.33. * Update org.jglobus.gridftp and org.jglobus.myproxy libraries to version 2.1.0. --- Thu June 4 12:35:00 PST 2020 Tony: portal-R43.32. * Patches provided by Mark M and patches from Tony C. --- Wed June 3 15:25:00 PST 2020 mzhuang: portal-R43.31. * Require email confirmation before first login after registration. --- Thu May 28 16:20:00 PST 2020 Tony: portal-R43.30. Added CIPRES Share (FolderShare) module. --- Thu May 28 15:55:00 PST 2020 Tony: portal-R43.29. Modified UsageLimitSetter class to output more debugging messages. Wed May 27 09:10:00 PST 2020 Tony: portal-R43.28. Patches provided by Mark M. Sat May 23 11:05:00 PST 2020 Tony: portal-R43.27. Patches provided by Mark M. Thu May 21 12:40:00 PST 2020 Tony: portal-R43.26. Added two options in UsageLimitSetter class to set values for ‘xsede_nonus_su_limit’ and ‘other_nonus_su_limit.’ Tue May 12 10:05:00 PST 2020 Tony: portal-R43.25. Patches provided by Mark M. Thu May 07 11:20:00 PST 2020 Tony: portal-R43.24. Remove country list from registration page and page auto-refresh JS from dataList.jsp. Thu Apr 30 13:09:00 PST 2020 mzhuang: portal-R43.23. Admin Interface. Tue Apr 07 9:45:00 PST 2020 Tony: portal-R43.22. Queries efficiency improvements + patches provided by Mark M. Wed Apr 01 14:10:00 PST 2020 Tony: portal-R43.21. Reschedule XSEDE Gateway Attribute reporting job to avoid loops. Wed Apr 01 12:50:00 PST 2020 Tony: portal-R43.20. Added DATE_STARTED column at job_stats table. Mon Mar 23 13:00:00 PST 2020 Tony: portal-R43.19. Removed unnecessary error message in SubmitXsedeGatewayAttributeJob class. Mon Mar 23 10:08:00 PST 2020 mzhuang: portal-R43.18. Make cloud eligible flag survive re-start and fix MrBayes restart problem. Thu Mar 19 11:55:00 PST 2020 Tony: portal-R43.17. Performance improvements in XSEDE Gateway Attribute Reporting procedure. Tue Mar 17 19:50:00 PST 2020 Tony: portal-R43.16. See detail description in SVN rev. 2232. Mon Mar 16 10:38:00 PST 2020 Tony: portal-R43.15. Performance improvements in XSEDE Gateway Attributes reporting package. Mon Mar 16 09:05:00 PST 2020 Tony: portal-R43.14. Added new improved version of XSEDE Gateway Attribute Report module. Thu Mar 12 23:23:00 PST 2020 Tony: portal-R43.13. Added status report to XSEDE Gateway Attribute Report module. Fri Feb 21 09:02:00 PST 2020 mzhuang: portal-R43.12. Send BEAST CPU jobs and BEAST2 GPU/CPU jobs to AWS. Thu Feb 20 10:25:00 PST 2020 Tony: portal-R43.11. Patches provided by Mark M. Wed Feb 12 20:25:00 PST 2020 Tony: portal-R43.10. Patches provided by Mark M. Tue Feb 03 16:40:00 PST 2020 mzhuang: portal-R43.09. Cloud burst related changes. Tue Jan 07 09:50:00 PST 2020 Tony: portal-R43.08. Patches provided by Mark M. Fri Dec 16 09:28:00 PST 2019 mzhuang: portal-R43.07. Patches provided by Mark M. Fri Dec 13 10:22:00 PST 2019 Tony: portal-R43.06. Patches provided by Mark M. Tue Dec 10 09:40:00 PST 2019 Tony: portal-R43.05. Patches provided by Mark M. Thu Dec 05 10:15:00 PST 2019 Tony: portal-R43.04. Patches provided by Mark M. Force terminate the BEAST/BEAST2 data file parser if it runs more than 5 minutes. Tue Dec 03 09:40:00 PST 2019 Tony: portal-R43.03. Fixed bugs in XSEDE Attribute Reporting and BEAST/BEAST2 parser. Tue Nov 26 15:45:00 PST 2019 Tony: portal-R43.02. Fixed XML filer reader opening and closing issue. Mon Nov 25 16:45:00 PST 2019 Tony: portal-R43.01. Patches provided by Mark M. Mon Nov 25 13:15:00 PST 2019 Tony: portal-R43.00. New branch which contains BEAST/BEAST2 data file parsing, XSEDE Gateway Attribute reporting, and bug fixes and performance improvements. Fri Nov 22 09:40:00 PST 2019 Tony: portal-R42.18. Patches provided by Mark Z; when country code is empty, portal finds out the country from user's IP address. Wed Nov 20 13:10:00 PST 2019 Tony: portal-R42.17. Patches provided by Mark M. Mon Nov 04 20:00:00 PDT 2019 Tony: portal-R42.16. Patches provided by Mark M. Thu Oct 31 10:00:00 PDT 2019 Tony: portal-R42.15. Patches provided by Mark M. Mon Oct 28 14:40:00 PDT 2019 Tony: portal-R42.14. Patches provided by Mark M. Mon Oct 28 14:25:00 PDT 2019 Tony: portal-R42.13. Patches provided by Mark M. Wed Oct 23 15:30:00 PDT 2019 Tony: portal-R42.12. Patches provided by Mark M. Wed Oct 23 09:35:00 PDT 2019 Tony: portal-R42.11. Patches provided by Mark M. Fri Oct 11 20:35:00 PDT 2019 Tony: portal-R42.10. Patches provided by Mark M. Thu Oct 03 18:20:00 PDT 2019 Tony: portal-R42.09. Patches provided by Mark M. Sun Sep 15 15:00:00 PDT 2019 Tony: portal-R42.08. Patches provided by Mark M. Wed Sep 12 14:05:00 PDT 2019 Tony: portal-R42.07. Patches for CAS login bugs. Sat Sep 07 16:00:00 PDT 2019 Tony: portal-R42.06. Patches provided by Mark M. Thu Sep 05 14:55:00 PDT 2019 Tony: portal-R42.05. Patches provided by Mark M. Tue Sep 03 9:50:00 PDT 2019 Tony; portal-R42.04. Patches provided by Mark M. Tue Aug 20 7:00:00 PDT 2019 Tony: portal-R42.03. Patches provided by Mark M. Mon Aug 08 12:50:00 PDT 2019 Tony: portal-R42.02. Patches provided by Mark M. Mon Aug 05 13:00:00 PDT 2019 Tony: portal-R42.01. Patch for BEAST, provided by Mark M. Tue Jul 23 12:20:00 PDT 2019 Tony: Started a new branch 'portal-R42'. Branch portal-R42 has Paul's 'restart' function and Tony's BEAST data file parsing. Wed Jun 5 18:56:55 PDT 2019 Paul: portal-R41.21. Patch for beast2, provided by Mark Tue Jun 4 19:43:05 PDT 2019 Paul: portal-R41.20. Patch for beast2, provided by Mark Wed May 22 10:58:19 PDT 2019 Paul: portal-R41.19. Patches for beast and beast2, provided by Mark Mon May 20 09:23:56 PDT 2019 Paul: portal-R41.18. Patch for beast_tg, provided by Mark Fri May 17 11:07:00 PDT 2019 Tony: Patched CreateTask.run() method to enforce SU usage. Thu May 16 08:39:12 PDT 2019 Paul: portal-R41.16. Unspecified patches, provided by Mark Thu Apr 4 08:20:22 PDT 2019 Paul: portal-R41.15. Patch for paup, provided by Mark Thu Mar 21 08:14:43 PDT 2019 Paul: portal-R41.14. Patch for mafft, provided by Mark Tue Feb 26 08:29:28 PST 2019 Paul: portal-R41.13. Patch for BEAST2, provided by Mark Tue Feb 19 09:52:39 PST 2019 Paul: portal-R41.12. Patches to Bali-Phy, IQ-Tree, and Exabayes, provided by Mark Tue Jan 22 15:02:00 PST 2019 Tony: portal-R41.11. Added “#SBATCH --mem=25G” in test_comet_submit.py and comet_submit.py per Wayne's request. Thu Jan 10 13:21:06 PST 2019 Paul: portal-R41.10. Unspecified changes provided by Mark Wed Dec 19 21:47:00 PST 2018 Tony: portal-R41.09. Return descriptive error message when user tries to create a guest account. Previously the link was removed and user gets HTTP 404 error. Removed hardcoded link in the email sent to user when the user resets the password. Marked American Samoa, Puerto Rico, Guam, and Virgin Islands the same as US when calculating SU usages. Fixed username format check error during registration. Wed Dec 05 10:33:00 PST 2018 Tony: portal-R41.08. Removed guest logins to portal. Thu Nov 29 08:41:19 PST 2018 Paul: portal-R41.07. Patch for iqtree, provided by Mark Tue Nov 27 10:35:00 PST 2018 Tony: portal-R41.06. Updates to correctly calculate SU_COMPUTED in job_stats table. Tue Nov 20 07:28:00 PST 2018 Paul: portal-R41.05. Patch for beast2, provided by Mark Thu Nov 15 09:05:37 PST 2018 Paul: portal-R41.04. Patch for PAUP, provided by Mark Tue Nov 13 08:43:37 PST 2018 Paul: portal-R41.03. Patches for PAUP, ExaBayes, BEAST, and Partitionfinder2, provided by Mark Wed Nov 7 08:12:20 PST 2018 Paul: portal-R41.02. Patches for Paup, provided by Mark Mon Nov 5 21:29:20 PST 2018 Paul: portal-R41.01. upgraded Struts to version 2.3.36 and commons-fileupload to version 1.3.3 Tue Oct 23 09:45:00 PDT 2018 Tony: portal-R41.00. Removed bugs in tool registry. Thu Oct 18 09:45:00 PDT 2018 Tony: portal-R40.50. Fixed guest login and new registration bugs. Tony: Tool registry contains bugs. Do not use this revision. Tue Oct 16 21:35:00 PDT 2018 Tony: portal-R40.49. Updated methods to allow users to login with username or email. Tony: Tool registry contains bugs. Do not use this revision. Fri Oct 12 14:00:00 PDT 2018 Tony: portal-R40.48. Updated SU calculations and SU limits enforcing algorithms, and other minor improvements. Tony: Tool registry contains bugs. Do not use this revision. Thu Oct 11 08:59:58 PDT 2018 portal-R40.47. Patches for beast and new code, modeltestng, provided by Mark Mon Oct 1 21:38:48 PDT 2018 portal-R40.46. Patch for migraten provided by Mark Mon Sep 24 08:13:14 PDT 2018 portal-R40.45. Patches for beast, beast2, and exabayes provided by Mark Sep 14 22:43:36 PDT 2018 portal-R40.44. Patch for batch tp beast2 provided by Mark Mon Aug 27 09:03:30 PDT 2018 portal-R40.43. Patch for IQtree provided by Mark Wed Aug 22 09:29:54 PDT 2018 portal-R40.42. Patch for dppdiv provided by Mark, update for struts to version 2.3.35 Thu Aug 16 13:31:38 PDT 2018 portal-R40.41. Patches provided by Mark Thu Aug 9 18:07:50 PDT 2018 portal-R40.40. Patch provided by Mark Mon Jul 30 18:53:03 PDT 2018 portal-R40.39. Patch for IQ Tree provided by Mark Thu Jul 26 16:37:07 PDT 2018 portal-R40.38. Patch for the BEAST2 interface provided by Mark Wed Jul 18 08:50:57 PDT 2018 portal-R40.37. Patches for BEAST and BEAST2 provided by Mark Thu Jul 05 12:20:00 PDT 2018 portal-40.36. Patches provided by Mark and deployed by Tony. Thu Jul 05 10:45:00 PDT 2018 portal-40.35. Patches provided by Mark and deployed by Tony. Wed Jun 13 16:44:40 PDT 2018 portal-R40.34. Patches provided by Mark Wed Jun 6 12:26:55 PDT 2018 portal-R40.33. Patch for the mrbayesrestart interface, provided by Mark Tue Jun 5 08:33:53 PDT 2018 portal-R40.32. Bug fix for the previous patch, provided by Mark Fri Jun 1 08:22:01 PDT 2018 portal-R40.31. Patch provided by Mark Tue May 29 12:24:10 PDT 2018 portal-R40.30. New code trimal added by Mark Fri May 18 09:08:54 PDT 2018 portal-R40.29. Patch for raxml provided by Mark Mon Apr 30 20:30:04 PDT 2018 portal-R40.28. Patch for mafft provided by Mark Fri Apr 20 10:15:58 PDT 2018 portal-R40.27. Patch for Iqtree provided by Mark Wed Apr 11 10:01:20 PDT 2018 portal-R40.26. Patch for MrBayes restart provided by Mark Thu Apr 5 21:53:17 PDT 2018 portal-R40.25. Fix for an expired PAUP code in the interface provided by Mark Tue Feb 27 09:29:46 PST 2018 portal-R40.24. Unspecified patches provided by Mark Wed Feb 21 09:02:27 PST 2018 portal-R40.23. Patch for Iqtree provided by Mark, separated killing and deleting jobs Tue Feb 20 08:49:29 PST 2018 portal-R40.22. Patch for Iqtree provided by Mark Fri Feb 9 08:34:57 PST 2018 portal-R40.21. Patches for raxml, clustal, fasttree and Iqtree provided by Mark Tue Jan 30 13:02:07 PST 2018 portal-R40.20. Patches for Partitionfinder2 and BEAST2 provided by Mark Tue Jan 23 21:08:40 PST 2018 portal-R40.19. Patch for Garli provided by Mark Thu Jan 11 10:59:11 PST 2018 portal-R40.18. Patch for raxml provided by Mark Mon Jan 8 08:19:13 PST 2018 portal-R40.17. Patch for raxml interface provided by Mark Wed Jan 3 14:00:29 PST 2018 portal-R40.16. Updates to PAUP provided by Mark Thu Dec 14 08:57:10 PST 2017 portal-R40.15. Patch for partitionfinder2 provided by Mark Sat Dec 9 08:49:13 PST 2017 portal-R40.14. Bali-Phy interface added by Mark Fri Dec 1 13:22:55 PST 2017 Paul portal-R40.13. Mark added a new tool, IQ Tree Wed Nov 1 08:43:44 PDT 2017 Paul portal-R40.12. Patches for phylobayes and raxml provided by Mark Tue Oct 24 09:17:50 PDT 2017 Paul portal-R40.11. Patch for partitionfinder2 provided by Mark Tue Sep 12 09:43:14 PDT 2017 Paul portal-R40.10. Mark forgot to remove a tool Tue Sep 12 09:13:22 PDT 2017 Paul portal-R40.9. Fixes for raxmlhpc8, updated Struts version Mon Jul 31 11:42:29 PDT 2017 Terri portal-R40.8. Another BlastN fix. Sat Jul 29 06:53:14 PDT 2017 Terri portal-R40.7. BlastN fix. Wed Jul 26 11:08:54 PDT 2017 Paul portal-R40.6. Fix for constraint violations when saving FolderItems. Mon Jul 17 08:28:21 PDT 2017 Terri portal-R40.5. Mark, unspecified changes. Fri Jul 14 08:57:47 PDT 2017 Terri portal-R40.4. Mark updated paup and beast2 changes. Fri Jul 7 09:49:11 PDT 2017 Terri portal-R40.3. Mark updated paup binaries. Mon Jul 3 09:28:24 PDT 2017 Terri portal-R40.2. Update accounting.period.start build property to 7/1/2017. Added it to build.properties file. Tue Jun 13 13:14:11 PDT 2017 Terri portal-R40.1. Mike Dwyer just changed our apache config to support CAS with iplant/cyverse instead of shibboleth. This new branch (40) has corresponding cipres code changes. Mon Jun 12 13:30:21 PDT 2017 Terri portal-R39.5. Mark pfinder. Tue Jun 6 11:35:47 PDT 2017 Terri portal-R39.4. Mark raxml v8. Fri Jun 2 15:56:37 PDT 2017 Terri portal-R39.3. Mark changed partionfinder2. Thu Jun 1 12:57:03 PDT 2017 Terri portal-R39.2. Mark changed garli. Wed May 31 09:11:54 PDT 2017 Terri portal-R39.1. Edit documents. Max size to view/edit is set to 5meg in default properties. I also had to set maxPostSize to 5 meg (plus some, for the rest of the form) in $CATALINA_HOME/conf/server.xml for each connector we use. In this case that's just the ajp connector. Tue Apr 25 11:00:12 PDT 2017 Terri portal-R38.24. Beast2 Fri Apr 14 06:59:20 PDT 2017 Terri portal-R38.23. Treeannotator Thu Apr 13 13:14:13 PDT 2017 Terri portal-R38.22. Beast2 Tue Apr 11 14:31:24 PDT 2017 Terri portal-R38.21. Fixed duplicate error messages and got rid of some registration failures due to using RegisterUser-validation.xml. No longer using xml file. Tue Apr 11 09:08:29 PDT 2017 Terri portal-R38.20. raxmlhpc8 Mon Apr 10 10:44:27 PDT 2017 Terri portal-R38.19. paup Thu Apr 6 10:14:56 PDT 2017 Terri portal-R38.18. beast2 Mon Apr 3 10:46:22 PDT 2017 Terri portal-R38.17. mrbayes restart Mon Apr 3 10:46:22 PDT 2017 Terri portal-R38.16. mv phylobayes to comet. Mon Mar 27 11:14:42 PDT 2017 Terri portal-R38.15. garli patches. Thu Mar 23 07:29:54 PDT 2017 Terri portal-R38.14. raxml workflow patch Mon Mar 20 10:55:28 PDT 2017 Terri portal-R38.13. Moved all tools off gordon -> comet. Thu Mar 9 18:52:25 PST 2017 Terri portal-R38.12. Changed Xsede password and rebuilt. Thu Mar 9 11:42:37 PST 2017 Terri portal-R38.12. Upgrade struts to 2.3.32. Thu Mar 2 13:21:00 PST 2017 Terri portal-R38.11. Garli fix again. Also, changed build.properties so that upload file size limit is now 2G. Up from default value of 6Meg, I think. Mon Feb 27 08:50:32 PST 2017/ Terri portal-R38.10. Garli fix. Fri Feb 3 16:17:34 PST 2017 Terri portal-R38.9. Some bug fixes and a mbayes_restart fix. - update to EnforceUniqueLabels to escape special chars in usernames when looking for a user's home folder. - backed out Pauls change in rest api that that I think caused Task StaleRowException Tue Jan 31 07:53:45 PST 2017 Terri portal-R38.8. garl2 fix. Fri Jan 27 08:22:12 PST 2017 Terri portal-R38.7. mrbayes_restart fix. Thu Jan 26 15:24:54 PST 2017 Terri portal-R38.6. mafft_xsede change. Tue Jan 24 10:25:22 PST 2017 Terri portal-R38.5. Paul made a couple of fixes in FolderItem.java. Sat Jan 21 06:16:37 PST 2017 Terri portal-R38.4. Mark commented out tools that should not have been in the registry. Wed Jan 18 13:56:29 PST 2017 Terri portal-R38.3. Fixed deleteUserData.py to deal with fact that all users have a home dir now (with label = username). Will re-enable it in cron now. Wed Jan 18 07:11:52 PST 2017 Terri portal-R38.2. Mark fixed beast and mafft. Mon Jan 16 13:52:24 PST 2017 Terri portal-R38.1. New branch, lots of changes, database schema changes too. Thu Dec 22 16:20:50 PST 2016 Terri portal-r37.32. Mark, update parallel structure. Thu Dec 22 13:09:51 PST 2016 Terri portal-r37.31. Mark, added group element(s)(?) to pise file try to fix command line ordering for mafft. I am also going to try building with java8 for the first time here. Wed Dec 21 14:37:32 PST 2016 Terri portal-r37.30. Mark, new version of mafft. Mon Dec 19 08:43:01 PST 2016 Terri portal-r37.29. Mark, fixes to beast, beast2, raxmlhpcc2tgb. Tue Dec 13 09:59:57 PST 2016 Terri portal-r37.28. Mark, parallel structure. Wed Nov 16 14:12:37 PST 2016 Terri portal-r37.27. Mark, paup. Mon Oct 31 06:20:35 PDT 2016 Terri portal-r37.26. Paul added UUID col to Folder.java. Pick up this code change and run his sdk/scripts/database/10*.py script to change the db schema. Thu Oct 13 14:47:18 PDT 2016 Terri portal-r37.25. Mark tool patches Thu Oct 6 09:28:50 PDT 2016 Terri portal-r37.24. Mark, moved raxml workflow to gordon. Wed Oct 5 13:56:27 PDT 2016 Terri portal-r37.23. Mark, fastree changes Fri Sep 30 14:05:02 PDT 2016 Terri portal-r37.22. Mark moved some tool back to gordon. Fri Sep 30 14:05:02 PDT 2016 Terri portal-r37.21. Modify db by running 07_19_2016.py which adds a UUID column to userdata. Made corresponding change to UserDataItem.java. Can't go back to running older tags without dropping the UUID column. (The code that really needs this col is in the rest api, on the trunk, not here, so if we really need to drop the col and go back to an older version it's ok). Tue Sep 13 12:51:23 PDT 2016 Terri portal-r37.20. Beast2 fix Tue Sep 6 17:43:22 PDT 2016 Terri portal-r37.19. Beast fix Wed Aug 10 16:20811 PDT 2016 portal-r37.18. Mark fixed raxmlhpc8. Tue Jul 12 12:44:01 PDT 2016 portal-r37.17. Mark fixed treeannotator. Fri Jul 8 08:42:29 PDT 2016 portal-r37.16. Mark fixed migrate-n. Wed Jul 6 12:19:54 PDT 2016 Terri portal-r37.15. Mark added a couple of tools. Mon Jul 4 11:00:19 PDT 2016 Terri portal-r37.14. update accounting period build property. Mon Jun 6 08:45:17 PDT 2016 Terri portal-r37.13. pauprat fix. Fri Jun 3 16:51:17 PDT 2016 Terri porta-r337.12. Mark made fix to fasttree. Tue May 24 09:50:15 PDT 2016 Terri portal-r37.11. Mark added new mafft options. Mon May 2 09:29:40 PDT 2016 Terri portal-r37.10. Missing comma in allowed-methods for "task" action in struts.xml. Prevented users from being able to display 2nd (and subsequent pages) of list of tasks. Thu Apr 21 13:23:49 PDT 2016 Terri portal-r37.9. Mark changed raxml to version 8.2.8. Tue Apr 12 17:11:12 PDT 2016 terri portal-r37.8. Previous tag was messed up. Had forgotten to comet y raxml_workflow pise changes to the branch. Mark copied it over from trunk for me. Tue Apr 12 07:23:37 PDT 2016 Terri portal-r37.7. Fix to raxml_workflow to use multiple srun commands. raxml_workflow puts new property in scheduler.conf: "workflow_type=simple" to trigger comet_submit.py to do this. Wed Mar 16 16:32:14 PDT 2016 Terri portal-r37.6. Build failed. Had to add some properties to build.properties to go along with the config web app name changes from Paul's stuff. Wed Mar 16 16:32:14 PDT 2016 Terri portal-r37.5. Upgrade to latest version of struts 2.3.24.1. Also discovered that Paul's changes for configurable web appnames committed on 2/10 didn't make it over here so copied them from trunk: default/build.properties.template and build.py. Fri Feb 19 12:40:11 PST 2016 Terri portal-r37.4. Beast 1.8.3 Tue Jan 19 07:58:26 PST 2016 Terri portal-r37.3. Change max upload size to 600 meg in 2 places in struts.xml. Also changed so this is controlled by a build property, currently defined only in defaults. Mon Jan 11 09:49:44 PST 2016 Terri portal-r37.2. Beast2 change. Tue Dec 15 14:45:31 PST 2015 Terri portal-r37.1. New branch from trunk, mostly for restructuring of rest examples. Fri Dec 11 16:40:44 PST 2015 Terri portal-r36.8. Changes to blastn. New db option. Wed Dec 2 08:58:07 PST 2015 Terri portal-r36.7. Changes to beast 1.8.0 Mon Nov 16 14:09:43 PST 2015 Terri portal-r36.6. Fixed build.properties file we copy in and copied it to this tag directory and re built. Somehow a space was introduced after max.file.size which ruins the parsing. I'll fix the code later to trim the string. Fri Nov 13 09:36:58 PST 2015 Terri portal-r36.6. Mark says minor raxml bug fix. Wed Nov 4 07:35:23 PST 2015 Terri portal-r36.5. Fix for the way we run pise on gpu. Wed Oct 28 14:04:33 PDT 2015 Terri portal-r36.4. Mark's beast update. Paul fix "cite us" link. Fri Oct 23 15:14:20 PDT 2015 Terri portal-r36.3. Mark fixed raxml to not let user specify more than 168 hrs. Wed Oct 7 13:41:23 PDT 2015 Terri portal-r36.2. Added fasttree_2.8.1 Mon Oct 19 19:11:18 PDT 2015 Terri portal-r36.1. Had to abandon r35 branch since Mona's changes weren't committed yet when I made it. Mona's changes: 1. Display user data size on folders panel on lhs of screen. Warn when limit reached (see user.data properties added to default properties file). Size display is refreshed on login and when data is deleted. What about when it's added? User preferences are used to disable job submission when user is over limit. 2. When archiving task for download, archive used to have duplicates of files. Mona fixed this. Tue Oct 6 14:25:22 PDT 2015 Terri portal-r35.1. New release from the trunk, includes my changes and Mona's. Mine were to add code to email cipres admins when there are errors submitting to the backend. Mona's were ??? Thu Oct 1 07:35:38 PDT 2015 Terri portal-r34.28. New versions of beast. New wrappers. Tue Sep 22 16:24:37 PDT 2015 Terri portal-r34.27. Updated raxml pise files to use new wrappers for new version of raxml: 8.2.3. Mon Aug 24 09:51:17 PDT 2015 Terri portal-r34.26. Terri Have submitJobD use submit_tools.log instead of tools.log. File was being corrupted because web app also writes to it. Keep an eye out for NullPointer exception in submit_tools.log on mrbayes jobs. Wed Jul 29 13:59:20 PDT 2015 Terri portal-r34.25. From Mark: patches to raxml workflow, beast, and garli2 xml docs. They are tested and checked in on branches 34 and r5. I updated the rest documentation for beast, the other canges did not affect documentation Thu Jul 16 07:43:52 PDT 2015 Terri portal-r34.24. Migraten and raxml changes. Wed Jul 1 10:06:19 PDT 2015 Terri portal-r34.23. Back to using our own allocation on xsede. Changed registry entry for coemt. Wed Jul 1 08:05:33 PDT 2015 Terri portal-r34.22. Update xsede accounting period start date. Mon Jun 29 18:57:16 PDT 2015 Terri portal-r34.21. I think this is to run raxml node exclusive. Fri Jun 19 13:56:32 PDT 2015 Terri portal-r34.20. Mark made changes to beast/beast2. Tue Jun 16 08:50:43 PDT 2015 portal-r34.19. Move raxml to gordon. Committed registry with ddp156. Also, a change to mrbayes. Mon Jun 8 09:34:48 PDT 2015 Terri Temporarily change tool registry so that comet charges to ddp156. Not committing the change; I don't know how long we'll be using ddp156. ****** TODO: if we push something new to production while we still need to charge to ddp156, remember to make this change on the new tag (or commit it to the branch). ****** Thu May 28 07:04:21 PDT 2015 Terri portal-r34.18. Mark migtrate-n fix. Not sure what 34.17 was! Tue May 19 11:11:59 PDT 2015 Terri portal-r34.16. Checkjobs was failing to check comet with error message about fasttree_xsede not in registry. Added fastree_xsede back in registry with inactive=true flag set. Goofed - this is now portal-r34.17. Fri May 15 10:20:55 PDT 2015 Terri portal-R34.15. Mark fix to migrate-n. Thu May 14 09:02:24 PDT 2015 Terri portal-R34.14. Fixes to rules for running some tools. Changed a few pise xml files as well as back end code on comet and gordon. Tue May 12 12:56:40 PDT 2015 Terri portal-R34.13. Fixes to fasttree and garli_conf_creator Mon May 11 09:11:41 PDT 2015 Terri portal-R34.12. Migrate-n fix. Thu May 7 18:15:21 PDT 2015 Terri portal-R34.11. Mark added migrate-n. Not redeploying but committed a fix to tgusage.sh from trestles to comet and making the same change to deployed version under $SDK_VERSIONS. Be careful if redeploying from current or earlier tag; the change under $SDK_VERSIONS wil be lost. Fri May 1 08:31:25 PDT 2015 Terri portal-R34.10 redeploy with build.properties corrected comet_workspace instead of comet_test_workspace. Thu Apr 30 12:20:23 PDT 2015 Terri portal-R34.10. Trestles is being retired today. Mark switched the tool registry so that all the trestles tools now run on comet instead. Wed Apr 29 09:35:37 PDT 2015 Terri portal-R34.9. a. Paul on 4/28/2015 wrote: Looking through the log files on qball2, I found that both of these jobs encountered a bug in the new code that handles duplicate source document signatures. Basically, the original attempt to insert a new source document record was being repeated when it shouldn't have been, and was failing due to some of the fields having invalid values. I've fixed the bug and committed the fix to the trunk and the portal-R34 and rest-R5 branches. As regards the other missing documents, I'm not sure what's going on there. None of those source document records exist in the database anymore, so maybe the script examined them midway through deletion? b. I added logging to WorQueue.postWork so I can grep all-portal2.log for "POST WORK" to debug something - more details in WorkQueue.java commit comment. Mon Apr 27 11:04:10 PDT 2015 Terri portal-R34.8. Jmodeltest fix. Fri Apr 24 16:16:22 PDT 2015 Terri committed fix to usage.py to change the delimiter from comma to tab. Don't want to redeploy just for that so I'm also manually changing the copy in $SDK_VERSIONS. If we redeploy from an existing tag the change will be clobbered. Making a new tag will pick it up from svn though. Wed Apr 22 07:36:36 PDT 2015 Terri portal-R34.7. comment out migrate-n. Fri Apr 17 15:08:02 PDT 2015 Terri portal-R34.6. Mona's changes for multiple file upload. Thu Apr 16 17:21:02 PDT 2015 Terri portal-R34.5. Fix to update_crl.sh Tue Apr 14 07:45:31 PDT 2015 Terri portal-R34.4. Just changing build.properties iplant charge number to be the same as Cipres's charge number. Thu Mar 26 10:13:00 PDT 2015 Terri portal-R34.4. Mark found problem in raxmlhpc3_xsede. Somehow extraneous line breaks were introduced that break the javascript. Mon Mar 23 07:09:48 PDT 2015 Terri portal-R34.3 - no change in version. Removed ~/.m2/settings.xml. Build was picking up old xsede password (teragrid.password) from this file instead of using value from cipres-config/build.properties. Just rebuilding and deploying now. Fri Mar 20 11:33:24 PDT 2015 Terri portal-R34.3. Mark removed tools that should not have been in the registry: paup and migraten. Thu Mar 19 06:44:46 PDT 2015 Terri portal-R34.2. Mark 1) roll back version of mrbayes, 2)add versioning to beast with ctrl that lets the user choose the version. 3) beastwrapper becahse beastwrapper_1.8.0 beastwrapper_1.8.1. Mon Mar 16 14:39:31 PDT 2015 Terri portal-R34.1. New branch. Has numerous changes including new interface for raxml8 plus raxml8_worflow interface. Some fixes from Mona so country code is only required for STANDARD users. Thu Mar 5 08:40:32 PST 2015 Terri Not redeploying but corrected /fs/portal2prod/scigap to have tscc.workspace of tscc_workspace, not tscc_test_workspace, so next build will go back to using the correct production workspace. Thu Feb 19 07:39:14 PST 2015 Terri See Jan 12 change. I knew there was some reason we never do it that way. Tag portal-R32.14 doesn't have the importTgusage fix, though the fix is on the branch. Creating and deploying portal-R32.15 to pick up the tgusage fix. Thu Jan 29 09:28:16 PST 2015 Terri back to portal-R32.14 due to problem with admin accounts and registration. Wed Jan 28 15:16:29 PST 2015 Terri portal-R33.1. Made a new branch portal-R33. It has Mona's registration/profile changes that make country code mandatory. Mon Jan 12 11:57:52 PST 2015 Terri Modified $SDK_VERSIONS/importTgusage. I had accidentally broken this script and the fix was very simple. Committed fix to trunk, portal and rest production branches but instead of redeploying the whole site I just put the fixed version of the script in $SDK_VERSIONS and will manually run usage.sh right now to test and get the tgusage data into our db. Mon Dec 15 10:36:51 PST 2014 Terri R32.14. Mark raxml changes. Fri Dec 5 12:43:06 PST 2014 Terri portal-R32.13. Fixed broken staticSite links (XSEDE Primer, Fair Use, IPlant What's This?) on login page. Thu Dec 4 06:40:35 PST 2014 Terri portal-R32.12. Mark said his change to dppdiv didn't make it into the last tag somehow. Wed Dec 3 10:26:19 PST 2014 Terri portal-R32.11. Mark accidentally committed a prelim paup interface to registry. This tag removes it. Tue Dec 2 13:19:37 PST 2014 Terri tag portal-R32.10. Has changes to dppdiv. Fri Nov 28 11:12:56 PST 2014 Terri branch_R32. Switched over to new build system. The top of the source dirs is now ~/scigap instead of ~/cipres-portal-src. I had to deploy the branch because I couldn't make a tag here, I think because portal2prod doesn't have write permission for the repo. It's rev 151 for scigap repo and rev 8968 for cipres-config, cipres repo. Mon Oct 20 19:01:12 PDT 2014 Terri R31.17. Mark moved mafft from trestles to gordon. Mon Oct 13 13:33:20 PDT 2014 Terri R31.16. Mark fix to phylobase. Tue Oct 7 13:59:10 PDT 2014 Terri R31.15. In struts.xml increased max upload size from 350mb to 600mb. A user requested this. Not sure if we should leave it at this higher level. Changed on both the branch and trunk in case we do want to leave it. Tue Sep 23 10:07:03 PDT 2014 Terri R31.14. 1. Mark update to raxml. 2. Terri made email validation less restrictive. Wed Sep 17 07:49:33 PDT 2014 Terri R31.13. Mark and David added jModelTest2, a new code. Tue Sep 9 16:24:23 PDT 2014 Terri R31.12. Mark raxml fix. Wed Aug 27 08:51:13 PDT 2014 Terri R31.11. Mark tool fix. Fri Aug 22 16:40:33 PDT 2014 Terri R31.10. Mark fixed raxml, fogbugz id 354536 Tue Aug 19 11:18:36 PDT 2014 Terri R31.9. Should be the same as R31.8. Exceeding disk quota was the issue. Tue Aug 19 10:46:51 PDT 2014 Terri Redeploy R31.7 because Mark says beast and raxml are broken in a strange way on R31.8. Maybe because we just exceeded our disk quota? ALERT: Next time we make a new production branch from trunk (R31): THIS IS DONE. - need scheduled downtime to make db changes. - db changes: * removed users.appname, changed email index,added umbrella_appname. * added appname to job_stats and tasks. * Added job_stats.su_override column * Added application_preferences table. * run may_29_2014.sql to make these changes. - need cipres admin credentials in ~/.m2/settings.xml, they are used by restusers app, but I think causes parsing problems elsewhere if missing. "admin.username", "admin.password". - run testAndDemoAccounts to add guidemo2 - need to update static web page that explains cpu hr Mon Aug 18 13:45:40 PDT 2014 Terri R31.8. Deploy beast2_xsede.xml changes mark made back on 7/24. Mark said there are also some raxml pise changes. Thu Jul 24 12:26:58 PDT 2014 Terri R31.7. Mark fixed login username requirement that disallowed capital letters. Wed Jul 23 07:49:02 PDT 2014 Terri R31.6. I changed results too large message to include a link to download the results. Changed BaseProcessWorker.java. Tue Jul 22 15:06:26 PDT 2014 Terri R31.5. Mark updated raxml interface. Fri Jul 11 14:30:46 PDT 2014 Terri R31.4. Mafft threads_per_process = 8 instead of 32. Also changed submit.py on backends. Tue Jul 1 12:50:00 PDT 2014 Terri R31.3. Update accounting period start in sdk/scripts/usage.py and cipres-build/*.properties. Mon Jun 30 19:29:48 PDT 2014 Terri R31.2. Mark took old mrbayes out of registry? Something like that. Mon Jun 30 08:45:47 PDT 2014 Terri R31.1. Backed up db and ram may_29_2014.sql conversion script. Added admin credentials to m2/settings.xml. Asked Mark to update static web page that explains cpu hr. Mon Jun 23 14:43:08 PDT 2014 Terri R30.15. Mark fixed raxml precond error. Wed Jun 18 15:53:00 PDT 2014 Terri R30.14. More raxml files. version # issues. Mon Jun 16 19:03:55 PDT 2014 Terri R30.13. Paul committed a change to fix ant build problem - missing xpath for portal codegen. Had already committed it for sdk I think. Mon Jun 16 16:56:43 PDT 2014 Terri R30.12. raxml pise update. Mon May 19 09:48:06 PDT 2014 Terri R30.11. Moved database. Thu May 8 16:11:21 PDT 2014 Terri R30.10. Added -k to callback url to avoid cacert error. Thu May 8 15:09:31 PDT 2014 Terri R30.9. Password reset email link and message fix, made by Mark and/or Paul. Thu May 8 10:08:19 PDT 2014 Terri portal-R30.8. Mark fixed mb interface. Wed May 7 14:38:28 PDT 2014 Terri portal-R30.7. Changed callback url to https since curl (at least the way we're calling it) doesn't follow redirects and Paul and Mike just changed our apache httpd config to send a redirect for any non ssl requests (so http -> https in a browser). Mon May 5 09:53:10 PDT 2014 Terri portal-R30.6 Marks fasttree changes. Tue Apr 29 18:14:56 PDT 2014 Terri portal-R30.5. 1) Mark fixes to phylobayes and mafft pise. 2) Terri fixes to JobCharge. Previously we weren't moving jobs that were never started because jobcharge failed on missing start.txt. Also was reading strings wrong from text files resulting in garbage chars in laodResults.log. Mon Apr 28 12:14:56 PDT 2014 Terri portal-R30.4. Removed raxmllight from registry. LoadResults fails on it because of JobCharge and the fact that it doesn't producte start.txt. Thu Apr 24 15:31:52 PDT 2014 Terri portal-R30.3. Fixed save task output. Need hidden datatype fields in outputForm.jsp. Rizhen must have removed them? Tue Apr 22 12:48:02 PDT 2014 Terri portal-R30.2. Took sleep and terri-raxml out of tool registry. Thu Apr 17 13:32:32 PDT 2014 Terri Deployed portal-R30.1 - 1st time for this branch. - before deploying backed up db under ~terri/secure_scripts - also before deploying, added SU_* cols to job_stats, built sdk and started to run sdkrun org.ngbw.utils.MigrateDbDocuments. Restarted it after deploying and it's still running right now. - modified crontab since many things that were under ~scripts are now in $SDK_VERSIONS. Saved old crontab in home diretory as crontab_branch29.txt. - For XSEDE jobs run prior to this release we need to fill in job_stats.tg_charge_number and job_stats.computed columns. usage.py -j takes care of that. Wed Apr 16 15:19:03 PDT 2014 Terri Changed cipres xsede password in settings.xml and redeployed. Fri Mar 7 16:58:26 PST 2014 Terri R29.6. Mark added dppidiv back to registry. Fri Mar 7 08:03:42 PST 2014 Terri R29.5. Mark. phylobayes fix. Thu Mar 6 17:37:59 PST 2014 Terri R29.4. Mark changes to pisexml files. Thu Feb 13 11:39:42 PST 2014 Terri R29.3. Paul changed the locking in FileColumn to allow for greater concurrency. Thu Feb 13 11:25:34 PST 2014 Terri R29.2. Paul's fix is here, though it still can take proceses a very long time to get a directory on startup. Updated applications table in db. Thu Feb 13 10:34:41 PST 2014 Terri R28.7. Had to back out new branch because loadResults hangs in FileColumn.getNewFile. Doesn't seem to have Paul's fix for that. Note to myself: turns out maybe the fix is in both places, but 1) I removed the logging from the fixed code on the trunk and 2) the fix isn't helping much. I just watched it log checking for size exceeded on 10,000 dirs, taking about 30 min and still not finished. There's a huge problem here. Other note to myself: I still need to update the application table for the branch 29 code whenever we switch to branch 29. Only the rest code uses that table so it doesn't really affect the portal, but I want to keep the table structure consistent with the code that's deployed. Thu Feb 13 09:40:18 PST 2014 Terri R29.2. Fixed sdk/pom.xml, removing database.username = test user Thu Feb 13 08:24:25 PST 2014 Terri R29.1. This branch includes substantial changes to our source code directory layout and build scripts. It no longer uses SDK_VERSIONS/current, all that stuff is directly under SDK_VERSIONS now, so PATH needs to change and everything in SDK_VERSIONS needs to be removed when switching between this branch and earlier versions. This branch build also depends on virtualenv python. Saved old settings-qball.sh. Mon Feb 10 10:23:38 PST 2014 Terri R28.7. Mark's beast changes. Tue Jan 14 16:32:30 PST 2014 Terri R28.6. Mark updated raxml version number. Also changed runtime ctrl in raxmlhpc2bb.xml. Fri Jan 3 17:54:14 PST 2014 Terri R28.5. Mark's beast.xml changes, implemented around 12/18. Thu Dec 12 16:35:12 PST 2013 Terri R28.4. Phylobase fixes from Mark. Thu Dec 12 16:35:12 PST 2013 Terri R28.3. Mark put wrong property in scheduler.conf for raxml. This should have the fix. Thu Dec 12 16:35:12 PST 2013 Terri R28.2. Changes needed due to trestles software upgrade that requires use of ibrun instead of mpirun, at least for raxml, and specify ppn=32 to get the whole node. Bryan changed trestles/lib.py, maybe submit.py and hopefully the raxml wrapper? Mark raxml pise. Tue Dec 10 07:01:47 PST 2013 Terri R28.1. New branch. Has Bryan's submit_v2.py, etc compatibility scripts in place (and installed on the backends). Also has Paul's fix for mistake that made cloning store a copy of the input source docs. Wed Dec 4 15:12:47 PST 2013 Terri R27.8. Mark phylobase fix or backed out fix that wasn't ready? Wed Dec 4 13:42:40 PST 2013 Terri R27.7. Fix umask in daemonize so that working dirs aren't created with full permissions for everyone. We set umask for portal2prod user in .bashrc or a script it invokes and then we need to prevent daemonize from overriding that. Umask should be 007 or 002. Wed Dec 4 10:39:18 PST 2013 Paul R27.6. Forgot to increment a debugging count Wed Dec 4 10:23:00 PST 2013 Paul R27.5. A bandaid for the slow directory search problem while I explore a more permanent solution. Thu Nov 28 11:38:31 PST 2013 Terri R27.4. More debugging info. Thu Nov 28 10:58:24 PST 2013 Terri R27.3. More debugging messages in FileColumn. May also have Mark's raxml fix? Mon Nov 25 14:12:44 PST 2013 Paul R27.2. Added debugging messages to FileColumn in an attempt to find out why file creation is taking so long. Mon Nov 25 12:49:45 PST 2013 Terri R27.1. Did database conversion this morning. Also changed CATALINA_HOME to point to 7.0.47. Have temporarily disabled ajp connector while I test. Need to re-enable it in order for httpd to work as front end. Fri Nov 22 15:12:13 PST 2013 Terri R26.20. Same thing, again. Fri Nov 22 06:54:55 PST 2013 Terri R26.19. Mark fixed his prior fix? Thu Nov 21 15:17:01 PST 2013 Terri R26.18. Mark fixed phylobase. Wed Nov 20 13:15:28 PST 2013 Terri Installed the latest version of tomcat 7.0.47 and configured for production ports. Haven't tested but it's ready to go. Need to change CATALINA_HOME. Just to be clear, we're still using the older version of tomcat until we change CATALINA_HOME. Sat Nov 16 09:38:29 PST 2013 Terri R26.17. Mark fix for phylobayes? Wed Oct 30 15:20:05 PDT 2013 Terri R26.16. Mark says he updated bug tracker links. Thu Oct 24 10:48:11 PDT 2013 Terri R26.15. Mark changed bug tracker link. Mon Oct 7 07:15:27 PDT 2013 Terri R26.14. Mark changes to mb and mbrestart. Mon Oct 7 07:15:27 PDT 2013 Terri R26.13. Mark updated pise xml files. Wed Sep 11 11:19:11 PDT 2013 Terri R26.12. Mark's raxml patch. Fri Aug 30 13:58:48 PDT 2013 Terri R26.11 Mark modified dppdiv pise xml and I modified gordon_lib.py to set ppn correctly for dppdiv (and for other tools. Change necessitated by OS/scheduler upgrade). Mon Aug 26 10:12:20 PDT 2013 Terr R26.10. Dppdiv change. Fri Aug 23 12:14:17 PDT 2013 Terri R26.9. Trying again. Mark said he forgot something. Fri Aug 23 11:06:23 PDT 2013 Terri R26.8. Mark's dppdiv changes. Tue Aug 20 14:26:27 PDT 2013 Terri R26.7. Added oldmrbayes to web app registry. Also Mark changed the best interface. Fri Aug 16 13:17:55 PDT 2013 Terri R26.6 Mark added new mrbayes tools and wanted to get rid of some old ones. We need all of them in the registry used by checkJobs and loadResults but only the new ones in the web app. I modified tl.sh to handle this. When the old tools are no longer needed we should copy tool-registry.cfg.xml.new over to tool-registry.cfg.xml. See the readme.txt in sdk/src/main/resources/tool/cipres. Wed Aug 14 12:24:33 PDT 2013 Terri R26.5. Mark asked me to redeploy to pick up changes to mrbayes. He says he is making MB 3.2 the default version. ??? Paul Hoover Paul said he changed cipres-portal/pom.xml and web.xml under tag_r26.4's directory on disk and deployed. He didn't commit the changes, just changed on disk. R26.4. Changed working dir locations on tscc from under home dir to under /projects. Also changed crontab on tscc to fix permissions and cleanup new working dir locations. Fri Jul 5 13:11:24 PDT 2013 Terri R26.3. Increase upload file size limit from 350mb to 750mb. Just changed in 2 places in cipres-portal/src/main/resources/struts.xml. Mon Jul 1 12:55:38 PDT 2013 Terri R26.2 Changes: 1. Paul fixed FileColumn issue that made it look like results were being truncated. 2. Paul's changes to run on tscc instead of triton. I believe he left triton resource config info in place, but configured all the triton tools to run on tscc instead of triton. 3. I think he also moved all the tools that used to run on snooker (tool.resource.host) to run on tscc instead. 4. Changed allocation period start date to 7/1/2013. 5. Raxml version number update. Thu Jun 20 07:29:11 PDT 2013 Terri R25.25. Mark is getting user reports of truncated job results so I'm rolling back to prior version for now. Tue Jun 18 14:22:01 PDT 2013 Terri R26.1. Made a new branch from trunk and commented out dppdiv because Mark says it isn't ready yet. Otherwise the purpose of this release is Paul's changes to the way we handle db FileColumn temp files. Paul also added a cron job to clean up the temp-* files. Part of the change is that we no longer copy tmp file to another file when saving the record - I believe the tmp file is just moved to it's permament location which is a much faster operation. No more holding a db connection open for hrs while the file is copied. Wed May 15 10:33:56 PDT 2013 Terri R25.25. Changed production database url (and db account info in settings.xml). New version of mysqld installed just for us. Fri May 10 15:03:49 PDT 2013 Terri R25.24. Beast upgrade to 1.7.5. Mon Apr 29 12:16:26 PDT 2013 Terri R25.23. Increased results limit to 3 gig, 5 gig, and changed the message that gets logged and emailed to users. (Can search loadResults.log for "ALERT"). Fri Apr 26 12:30:02 PDT 2013 Terri R25.22. Three primary changes: 1) Move results that are too large, in other words, if working dir has a single file > 2 gig, or sum of all files > 4 gig, to a MANUAL subdir. Stop automatic retries, set running task status to STOP_LOAD_RESULTS and send user a job completion email with msg that says contact customer service for help getting results. 2) Instead of using jdbc driver directly use log4jdbc which wraps the jdbc driver and logs info. So there's new info about connections and queries in the logs. 3) Use connection pooling in loadResults. If nothing else this may help with cases where the connection seems to have gone stale and mysqld may be happier if we don't open and close so many connections so quickly. Tue Apr 16 15:43:35 PDT 2013 Terri R25.21. See "DB TIME" in logs. Added some logging about how long it takes to get a connection, save a row. Also, test connections with mysql recommended query: /* ping */ instead of select 1. Mon Apr 15 15:33:32 PDT 2013 Terri R25.20 - repeat of previous. Mon Apr 15 15:33:32 PDT 2013 Terri R25.19. Changed db pool properties to account for queries taking a long time. Mon Apr 15 13:32:02 PDT 2013 Terri R25.18. Moved logs to a local filesystem: /fs/portal2prod/logs. Sun Apr 14 07:24:57 PDT 2013 Terri R25.17. In sdk/pom.xml upgraded to much more recent mysql jdbc Connector/J driver. Sat Apr 13 14:47:16 PDT 2013 Terri R25.16. Better debug msg when exception occurs that causes database rollback. Sat Apr 13 12:55:13 PDT 2013 Terri R25.15. Fixed SSHExecProcessWorker.readFile. Had forgotten to increment retries. Fri Apr 12 13:00:43 PDT 2013 Terri R25.14. Web app was reading intermediate results into memory. Fixed. Thu Apr 11 16:45:15 PDT 2013 Terri R25.13. Some loadResults fixes. Biggest is don't read stdout/stderr in memory. Commited pool size change below. Thu Mar 21 10:44:49 PDT 2013 terri R25.12. Editted in place (did not commit the change!) and deployed. Modified database.properties to set tomcatPool.initialSize to 1, instead of 10. I did this because LoadResults which doesn't use connection pooling, as well as mysql commonad line client were able to get single connections but the web app was failing with max_user_connections exceeded - I assume because for some reason it couldn't get 10 connections. Still not sure why this would be the case since we should be allowed 200 max_user_connections. Thu Mar 21 10:44:49 PDT 2013 terri R25.12. Raxml changes. Fri Mar 8 18:14:42 PST 2013 Terri R25.11. Raxml changes. Wed Feb 27 13:19:14 PST 2013 Terri R25.9. Not sure, Mark made changes to some tool interface, don't remember which. Mon Feb 18 09:20:01 PST 2013 Terri R25.8. Mark patched beast.xml. Fri Feb 1 15:29:43 PST 2013 Terri R25.7. Mark corrected tool registry. Previous release was bogus. Fri Feb 1 14:07:47 PST 2013 Terri R25.6. Mark added mrbayes 3.2.1 and mrbayes 3.2.1 restart. Old mrbayes is still there too. Wed Jan 30 13:45:21 PST 2013 Terri R25.5. Fix for OverlappingFileLock exception on job submission. See comment in common/util/GsiSSHProcessRunner.java. Wed Jan 23 12:54:21 PST 2013 Terri R25.4. Removed tool dppdiv. Tue Jan 22 11:51:24 PST 2013 Terri It looks like I didn't really deploy local file handler on 1/3 because between then and my deployment on 1/18, we created working dirs that are owned by cipres and are rwx------, owned by cipres. Sdk.log files show gridftp file handler created these dirs. We try to retrieve results with whatever file handler is currently specified by the tool registry so when we actually deployed local file handler on 1/18, portal2prod started trying to retrieve results with it but got permissions error for all the gridftp dirs that were owned by cipres. I just fixed permissions on those directories so recoverResults can take care of it. Tue Jan 22 08:42:36 PST 2013 Terri R25.3. bug fix for last change on raxml pise file. Tue Jan 22 07:26:09 PST 2013 Terri R25.2 Mark rolled back version of raxml referenced by the pise file. Fri Jan 18 15:48:14 PST 2013 Terri R25.1. Added last_login column to user table and set to today's date for everyone. Code will keep track of last login date now. Can run the new deleteUserData.py to delete old user data whenever we're ready. Thu Jan 3 14:13:56 PST 2013 Terri R24.7. Using local file system for tresltes and gordon instead of gridftp. This required changes to: * to tool-regsitry.cfg.xml and SSHExecProcessWorker since both specify the file handler to use. * For the xsede hosts modified workbench.rc to set umask 0007 (don't mask out any owner and group permissions), RaXML-Light-1.0.9.sh, and submit.py, also to set umask for the scheduler stderr and stout files. * The web accounts (i.e. portal2prod, portal2test, terri, etc) and the xsede cipres account must belong to the same cipres group and should set umask 007 so that files and directories they create, particularly the job working directory that they create and stage to has full permissions for both owner and group. Permissions for the working dir area, SGID bit, umask on web and xede hosts, must give permissions to the group as well as the owner. * Created a link from /projects/ps-ngbt to /archive/science/ngbt on billiards and qball2 so that the working dirs can be accessed at the same path from web server and from xsede side. See my google doc and to be added programmer documentation about what was required. Tue Dec 18 16:36:37 PST 2012 Terri R24.6. Mark fixed some links in home.jsp Fri Dec 14 16:13:17 PST 2012 Terri R24.5. Changed database.properties. removeAbandonedTimeout was being set to 0, which I assume means, don't remove "abandoned" connections. Fri Dec 14 15:09:06 PST 2012 Terri R24.4. Minor bug fix to GsiSSHProcessWorker. Removed assertion in unlockCertFile. Tue Dec 4 13:57:36 PST 2012 Terri R24.3. Modified SimpleCred to not use saml asserts. Putting -ea back into CATALINA_OPTS to enable assertions. Mon Dec 3 10:59:59 PST 2012 Terri Got rid of the -ea for now. Apparently the saml token code triggers an assertion. Todo: I need to take out all that saml code. Mon Dec 3 10:59:59 PST 2012 Terri Changed scripts/portal2_prod_exp.sh to keep some logs 90 days instead of 30. Changed CATALINA_OPTS to include "-ea" so assertions should be enabled now. TODO: am seeing frequent unexpected exceptions in all-portal2.log in GsiSShProcessRunner.unlockCertFile, line 115. Previous line should have thrown exception. Maybe the assertion will help me track it down. Check logs. Wed Nov 28 13:24:24 PST 2012 Terri R24.2. convertEncoding.sh fix for xml files that don't specify encoding. Mon Nov 26 15:28:37 PST 2012 Terri R24.1. BIG_FILES, connection pooling. Tue Nov 20 13:11:38 PST 2012 Terri R23.24. Changed message in BaseProcessWorker.java. "Check back in 8 hrs" instead of 24 hrs. Also changed crontab entry to run recover_results.sh every 8 hrs. Tue Oct 30 14:05:51 PDT 2012 Terri R23.23. Restored raxmlhpc2bb_abe.xml so that old jobs are displayed right in the task list. Tue Oct 30 12:18:31 PDT 2012 Terri R23.22. fix for stackoverflow and jvm crashes related to stack overflow. See NgbwSupport.getConceptLabel() comment for details. Fix for old jobs like raxmlhpc2_abe and raxmlhpc missing input files. Changed JAVA_HOME to use new version of 64 bit java (Andrew patched yesterday). Fri Oct 26 08:15:36 PDT 2012 Terri R23.21. mark patch to raxml. Thu Oct 25 10:02:47 PDT 2012 Terri R23.20. Mark fix for pauprat. I added resource mapping for FASTOUT to see if that will reduce some of the StackOverflow crashes. Fri Oct 19 16:21:45 PDT 2012 Terri R23.19. Going back to java 1.5. We're seeing frequent StackOverflowError and I want to see if the version of java matters. If not I'll have to back out some changes. Oct. 22 update: no more stack overflow messages than usual. Boy this is going to be a problem if we can't upgrade to java 1.6. Maybe I just need to specify a larger stack size - maybe the default changed between 1.5 and 1.6? Oct 22 update: had accidentally picked up 32 bit java from /etc/alternatives. I'm redeploying now with 64 bit java from /usr/local/apps. Thu Oct 18 10:09:00 PDT 2012 Terri R23.19. No code changes, but this time with java 1.6 instead of 1.5. This is the first time we're deploying with 1.6. Thu Oct 18 09:43:31 PDT 2012 Terri R23.19. Fix paste data which I broke with the memory debugging. Wed Oct 10 09:05:31 PDT 2012 Terri R23.18. Increased session timeout in web.xml to 90 min, for large file uploads on slow connections. Also dump amount of memory used/avail right before and after upload and reading file into memory. Mon Oct 8 16:04:42 PDT 2012 Terri R23.17. Some file upload memory use logging. Also display an error message when we run out of memory on file upload. Mon Sep 24 18:57:08 PDT 2012 Terri R23.16. raxml light and beast/beagle changes, trying again. Mon Sep 24 17:55:28 PDT 2012 Terri R23.15. Updates to raxml-light and beast/beagle. Corresponding changes on backend scripts too. NO!!! Had to back this out due to a build failure so we're currently on R23.14. Fri Aug 24 18:46:31 PDT 2012 Terri R23.14. Fixed saving of task label/description. Tue Aug 21 17:19:03 PDT 2012 Terri R23.13. Modified NgbwSupport.java and CreateTask.java to fix one dup task id scenario where user views task output, then goes to toolkit tab to start a new task and ends up re-using the id and data from the just viewed task. Thu Aug 16 16:31:08 PDT 2012 Terri R23.12. Changed default recoverResults period to 15 days since we only store working dirs on trestles and gordon for 14. Also, don't keep adding * for job_stats.stage when task is deleted. Thu Aug 16 15:30:23 PDT 2012 Terri R23.11. Debugging logging for recoverResults. Thu Aug 16 07:12:41 PDT 2012 Terri R23.10. Bug tracker back online. Mark changed link. Mon Aug 13 16:14:03 PDT 2012 Terri R23.9. Dump request headers on login so that we can see the session id (JSESSIONID). I'm also Turning on access logging in catalina (conf/server.xml accesslog valve) with session id displayed along with each url requested. Sun Aug 12 16:56:25 PDT 2012 Terri R23.8. Bug tracker broken, Mark put in message to users to email him. Fri Aug 10 14:36:17 PDT 2012 Terri R23.7. Log client ip address with username, right after login, so we can cross reference with apache access logs. Mon Jul 30 14:57:18 PDT 2012 Terri R23.6. A bunch of fixes to prevent double submit on task creation page. Also added some synchronization to Workbench.saveAndSubmit. Tue Jul 24 17:32:41 PDT 2012 Terri R23.5. Added check for Task Id already in RunningTask table in Tool.java Tue Jul 24 17:15:28 PDT 2012 Terri R23.4. More debugging. Forgot to commit it all last time. Sun Jul 22 11:42:41 PDT 2012 Terri R23.3. More task debugging. Added task id to "Multiple tasks with same id" message. Sat Jul 21 14:57:41 PDT 2012 Terri R23.2. Just added some debugging of task inserts and updates. Fri Jul 6 14:21:04 PDT 2012 Terri R23.1. Raxml-lite. Also Mark's changes to make mrbayes_xsede run on gordon and stop running mrbayes on trestles. Thu Jul 5 11:16:35 PDT 2012 Terri R22.6 Updated allocation period start to 2011-07-01. I think Mark made a couple of raxml interface changes. Thu Jun 7 11:41:51 PDT 2012 Terri R22.5. New versions of raxml and beast. Thu May 31 14:02:48 PDT 2012 Terri R22.4 Increased runtime on gordon to 2 weeks. Mon May 21 09:24:34 PDT 2012 Terri R22.3 Added Gordon as a resource and Mrbayes on Gordon as a tool. This uses a special ddp allocation, not the normal cipres allocation, though gsissh and gridftp are using the cipres credential. Mon May 14 10:07:52 PDT 2012 Terri R22.2. Commented out raxml-light. We should never deploy raxml-light on this branch. The changes it needs are still only on the trunk (probably will be on branch 23 as well). Fri May 11 13:12:06 PDT 2012 Terri R22.1. First release with iplant login. This required reconfig of apache and install of shibboleth on webfarm. Wed Apr 11 14:44:59 PDT 2012 Terri R21.6. Changed working dir file system to /projects/ps-ngbt/backend/trestles_workspace. Changed crontab to clean up this directory along with the previous one. Link to the working dir is ~/ngbw/workspace3. Wed Mar 21 12:37:32 PDT 2012 Terri R21.5. Mark forgot to commit last time. Wed Mar 21 09:31:12 PDT 2012 Terri R21.4. raxml 7.3, garli interface, beast interface bug fix. Thu Mar 1 10:34:23 PST 2012 Terri R21.3. Mark removed a couple of tools that shouldn't have been deployed. Tue Feb 28 13:40:04 PST 2012 Terri R21.2. Just changed tgusage.sh and needed to deploy it. Also changed usage.sh and usage.py which I manually copy to ~/scripts. We run this stuff on trestles now instead of dash and I keep track of cipres and iplant charge numbers separately in ~/tgusage. Fri Feb 24 14:23:01 PST 2012 Terri Started new branch R21 and deploying R21.1. Just had minor changes to support individual user allocations. User mlosada is now charging to his own allocation: TG-MCB120063. Mon Feb 20 13:40:12 PST 2012 Terri 20.15. (Problem with svn when deploying 20.14). Reduced thread pool size in sdk properties files for loadResults from 10 to 3 to see if this helps with the max_user_connections exceeded mysql error. Tue Jan 17 16:07:22 PST 2012 Terri 20.13. Mark changed mrbayes interface in response to a bug report. Fri Dec 16 09:42:17 PST 2011 Terri 20.12. Mark's changes. Wed Dec 7 10:24:30 PST 2011 Terri 20.11. Problem with Mark's prior commit. Trying it again. Wed Dec 7 09:43:24 PST 2011 Terri 20.10. Mark increased time allowed for mrbayes runs to 2 weeks. Thu Dec 1 12:59:49 PST 2011 terri 20.9. Mark says he added 2 new interfaces (tools?) Mon Nov 28 10:06:34 PST 2011 terri 20.8. Just increased max file upload size from 100meg to 350meg. Change is in struts.xml Made change on branch and trunk. Fri Oct 28 15:42:58 PDT 2011 terri 20.7. Previous version may not have actually had Mark's change committed. Fri Oct 28 15:42:58 PDT 2011 terri 20.6. Mark changed beast/beagle. Mon Oct 17 13:48:24 PDT 2011 terri Deploying 20.5. Mark made changes to beast. (On trestles, I just changed lib.py to allow runs up to 168 hrs in the shared queue). Tue Sep 27 15:12:53 PDT 2011 Terri Deploying 20.4. Adds tool "beast" Wed Sep 14 12:20:04 PDT 2011 Terri Deploying portal-R20.3. (not sure what 20.2 was). Moved workspace from /oasis to /phase1 on trestles. Thu Sep 1 11:09:40 PDT 2011 Terri Deploying portal-R20.1. Put garli_creator back. Thu Sep 1 07:46:02 PDT 2011 Terri Deploying portal-R20.1. Warns element added to code generation and certain pise files. Tue Aug 16 11:09:59 PDT 2011 Terri Deploying portal-R19.3. Just a minor change in SFTPFileHandler to set better default permissions on files and dirs we create. Tue Jul 26 10:49:53 PDT 2011 Terri Deploying portal-R19.2 - changed accounting period start to 8/1/2011 - removed quotes from date in the properties files, it's already quoted in User.java - broke up too long lines in scripts/alloc*.txt. Was causing ! to appear in the email. Mon Jul 25 17:05:59 PDT 2011 Terri New branch. Deploying portal-R19.1. - delete task now kills queued or running job (only on trestles) - statistics: many changes, including lookup by taskId AND jobhandle - SUs shown for each user for current period (given by property in cipres-prod.properties) - send email to user and Mark when too many Su's used. 10,000, 20,000 and 30,000 su limits. See sdk/scripts/usage.py, mailer.py, allocations*.txt/rtf, db_password.txt. Start of period is hardcoded in the scripts and should match date in cipres-prod.properties. Using scripts on trunk and copying the files to ~/scripts. Tue Jun 7 09:55:35 PDT 2011 Terri Mark's changes to mrbayes to eliminate 2 node runs. 18.14 Tue May 24 14:21:00 PDT 2011. Terri Deploying 18.13. Changes to tgusage.sh Fri May 6 10:53:43 PDT 2011 Terri Mark fixed limit in raxml. Deploying R18.12 Wed May 4 15:58:28 PDT 2011 Terri Mark fixed a bug in garli_tg interface. Deploying R18.11 Tue Apr 26 10:29:24 PDT 2011 Terri changed gridftp url for trestles on trunk and branch 18. Won't deploy until Mark has his garli changes ready, or whatever the next deployment reason is. Mon Apr 25 10:24:35 PDT 2011 Terri Deploy 18.10. raxml 7.2.8 instead of 7.2.7 running on trestles, changes to raxml pise files. Wed Apr 20 12:55:55 PDT 2011 Terri Deploy 18.9. Just changed the "How to cite us" link for Mark. Thu Mar 24 11:45:13 PDT 2011 Terri Deploy 18.8. Garli on trestles plus some changes Mark made to mafft interface. Mon Mar 21 11:28:46 PDT 2011 Terri Deploy 18.7. Just added tgusage stuff. Fri Mar 18 15:05:56 PDT 2011 Terri Deploy 18.6. Nothing running on abe anymore. Fri Mar 11 10:28:41 PST 2011 Terri Deploy 18.5, Mark's changes Tue Mar 1 12:49:00 PST 2011 Terri Deploy 18.4. Show teragrid and phylowidget icons on almost all pages. Fri Feb 25 14:27:45 PST 2011 Terri Deploy 18.3 to fix Statistics page. Was taking too long to load because of number of deleted tasks. Changed to show records even when task has been deleted. Thu Feb 10 06:42:37 PST 2011 Terri Deploy 18.2. Have mrbayes return _JOBINFO.TXT Thu Feb 10 05:39:43 PST 2011 Terri Deploy 18.1. Has mafft and mrbayes on trestles, in addition to raxml. Fri Jan 28 13:15:37 PST 2011 17.1 Thu Jan 27 10:03:27 PST 2011 back to 16.8, problem not fixed. Thu Jan 27 09:30:33 PST 2011 17.1 again, trestles support says submission error has been fixed. Wed Jan 26 14:24:10 PST 2011 Terri Going back to 16.8 because there's a problem submitting on trestles. Wed Jan 26 13:17:23 PST 2011 Terri Deploying 17.1. First release with trestles support. Raxml jobs on trestles. Wed Dec 15 16:56:04 PST 2010 Terri Deploying 16.8. CheckJobD was skipping all resources after deleted resource. Wed Dec 15 16:46:07 PST 2010 Terri Deploying 16.7. Paul made some fix to multiple file download. Mon Dec 13 13:40:35 PST 2010 Terri Deploying tag 16.6. Has Paul's fix for multiple file downloads. He renamed DataManager.getContentLength(). Said he also renamed it in derived classes. It was causing a problem in ArchiveManager because we start building and streaming the archive before we know it's length. Mon Dec 13 08:37:25 PST 2010 Terri Deploying 16.5. Intermediate results fixed. Wed Dec 8 10:20:09 PST 2010 Terri Deploying 16.4. Had to make changes to BaseProcessWorker.java so that we wouldn't try to call checkJobs and loadResults for tasks whose resource is no longer in the registry (i.e. lonestar). Wed Dec 8 08:30:56 PST 2010 Terri Deploying 16.3. Have upgraded from struts 2.0.9 to 2.1.1. This includes upgrade from ognl 2.6.11 to ognl 3.0 which may fix a "waiting for monitor" bug/deadlock we may have been seeing. Note that I had to clear out the whole ~/.m2/repository and let mvn download jars from scratch for this upgrade to take place. I've also removed lonestar from the registry. Remote invocation of checkjobs.py was hanging and we have no time-out on the call so checkJobsD had started it several times; Mark said the web site started allowing connections when he killed checkjobs.py processes so maybe this will help. We'll see. I also remember now that when I performed load testing of checkJobs and loadResults the c3p0 database connection pooling was clearly buggy wrt high contention situations and we are still using it in the web application. Mon Dec 6 09:25:57 PST 2010 Terri Lot's of problems with the portal freezing and user's seeing "Bad Gateway" since we deployed from branch 16. I'm rolling back to branch 13, specifically, I'm deploying 13.11, which should be the same as 13.10 except it includes Mark's most recent changes to clearcut and mafft. Thu Dec 2 10:42:59 PST 2010 Terri Just made branch R16 from trunk. Has all the R15 changes (user allocations, bug fixes, etc) plus Paul's bulk upload/dld features. Deploying R16.1 Thu Nov 18 14:11:46 PST 2010 Terri Data upload doesn't work. Going back to 13.10. Thu Nov 18 11:15:59 PST 2010 Terri Made new branch 15, deploying 15.1 Thu Nov 4 11:52:45 PDT 2010 Terri Deploying 13.10. Keep garli_on_lonestar as well as garli_on_abe. Tue Nov 2 11:58:24 PDT 2010 Terri Deploying R13.9. Fix to garli_on_abe (for 100 bootstrap case). Tue Nov 2 09:53:07 PDT 2010 Terri Deploying R13.8. Adds garli_on_abe tool. Tue Oct 12 12:35:25 PDT 2010 Terri Deploying R13.7. Bug fix for checkJobsD. When triton was down I noticed the problem that we'd get an exception running checkJobs for triton and the program would exit instead of continuing to check the next resource. This should fix it. Fri Oct 1 09:26:15 PDT 2010 Terri Deploying R13.6. I believe it has Mark's changes to raxml and garli interfaces. Wed Sep 22 05:50:07 PDT 2010 Terri I killed all process workers. They were all older than 14 days. We are no longer using DrmaaProcessWorkers and GlobusProcessWorkers. The only ones we should see anymore are short lived SSHProcessWorkers for running commands like nclconverter on bwbatch@snooker. Thu Sep 16 11:57:22 PDT 2010 Terri 13.5 Mark made a fix to mrbayes xml for bug in hybrid code Tue Sep 14 15:55:47 PDT 2010 Terri 13.4 Mark made a fix to raxml on abe pise xml. Tue Sep 14 09:58:52 PDT 2010 Terri 13.3 deployed. Only send job completion email when results have actually been transferred to the db. Installing recoverResultsD cron job to run once daily. I think the previous deployment must have actually been 13.2, not 13.3. Fri Sep 10 15:13:05 PDT 2010 Terri 13.3 deployed Fri Sep 10 12:58:00 PDT 2010 Terri deployed tag portal_R12.2. New submission mechanism for teragrid. Should have new "Science gateway" logo but doesn't. Submission (checkJobs) doesn't seem to be working right. Redeploying tag 11.6 Thu Jul 22 18:30:52 PDT 2010 Paul fixed code about disabled job submission. 11.6 Wed Jul 21 15:33:14 PDT 2010 Paul fixed code. Doing 11.5. Wed Jul 21 15:22:13 PDT 2010 Rolling back to 11.3. 11.4 had a build error. Wed Jul 21 13:01:30 PDT 2010 R11.4 to pick up Paul's changes for displaying status messages Wed Jun 23 13:51:26 PDT 2010 Terri Deploying R11.2, changes to scripts only. Thu Jun 17 12:48:16 PDT 2010 Terri Created branch R11 with new piseEval. Deploying R11.1 Mon Jun 7 11:35:28 PDT 2010 Terri Deploying from trunk, creating new branch R10, and first tag R10.1. This runs jobs on triton instead of cipres cluster. Mon Jun 7 around 7:30 PDT 2010 Terri Deployed from R8.12, commented pauprat out of registry. Should have Paul's change to allow larger file upload size. Tue Mar 30 11:19:30 PDT 2010 Terri We've moved from qball to qball2! Thu Mar 4 10:37:00 PST 2010 Terri Creating and deploying tag portal-R8.6. Should have Mark's fixes to running mrbayes on too many processors. Thu Feb 25 17:58:42 PST 2010 Terri Creating and deploying tag portal-R8.5 - Mark's got more changes to the tool interfaces. Tue Feb 23 14:32:15 PST 2010 Terri Created and deployed tag portal-R8.4, with Mark's changes to raxml and mrbayes interfaces. I also installed raxml 7.2.6 on abe which is what the raxml interfaces run now. Mon Feb 22 14:38:41 PST 2010 Terri Creating tag portal-R8.2 with changes to GlobusConfig, GlobusProcessWorker and added medium polling interval to cipres-*.properties. We're running out of disk space on lonestar partly because the gram_log* files are so big. Text is added each time we poll, so I've changed things such that once a few hours have gone by we only poll once an hour. Used to poll every 5 min. Thu Feb 11 09:37:32 PST 2010 Terri Modified $CATALINA_HOME/webapps/cipres-portal/pages/login.jsp - got rid of marquee "Announcing ..." and replace it with a message about planned downtime on 2/16/2010. Tue Jan 19 09:43:35 PST 2010 Terri Mark asked me to deploy to production changes he made to mrbayes interrace to add an autoclose field. I didn't test - just made the portal-R6.3 tag and deployed it. Fixed some problems on lonestar due to work filesystem crash. Portal depends on two directories under $WORK: ngbw/cipres/workspace and ngbw/cipres/globus. I had to recreate these. Mon Jan 11 06:47:32 PST 2010 Terri I accidentally committed a fix to mrbayes_abe.xml on tag-portal-R6.2 (instead of the branch). I redeployed from the updated tag. I'm also committing the change to branch-portal-R6 and the trunk. Tue Dec 8 09:56:58 PST 2009 Terri Removing non TG mrbayes and raxml on branch-portal-R6. I'll make tag_portal-R6.2 and deploy it here. Copying change to trunk. Tue Dec 1 14:38:26 PST 2009 Terri Added garli_lonestar on branch-portal-R6 and ran a few tests on portal2test. Creating tag-portal-R6.1 and deploying it here. (Am using new_wrap instead of wrap script on lonestar. Can always revert w/o code changes by swapping the scripts on lonestar if need be.) Just noticed that we're running tomcat 5.5.23 in test and 5.5.27 on production - would be better if they were the same. Mon Nov 30 11:14:17 PST 2009 Terri - Deploying tag_portal-R6, the first release with tools on teragrid and statistics. - Had to add statistics table to production db using documents/SWAMI_Design/statistics-table.sql and MySQL Query Browser. - Had to add cipres-admin role to tomcat-users.xml in CATALINA_HOME directory - To get basic auth for statistics page to work had to comment out a line in CATALINA_HOME/conf/server.xml: Thu Sep 17 08:44:51 PDT 2009 Terri Redeployed R5 with job submission disabled. I disabled it by modifying CreateTask.java. To re-enable, restore CreateTask.java from svn and redeploy. --- Sat Sep 19 05:51:58 PDT 2009 Terri Redeploying R5 with submission enabled. --- Mon Sep 21 12:05:54 PDT 2009 Terri * Logged into ```portal2prod@qball.sdsc.edu``` and am trying to switch over from using rhrel5-x86-testing to qball as our production server. We're still running on rhrel5-x86-testing and I want to bring the web app up on qball at the same time, so I'm temporarily changing the base dir we use to ```/archive/science/ngbt/portal2prod_qball```. * Deployed on qball and switched back to using the regular ```/archive/science/ngbt/portal2prod```. --- Fri Oct 9 09:12:13 PDT 2009 * Changing login.jsp under $CATALINA_HOME/webapps/portal2 to announce the planned downtime next week in the ticker. Not restarting the server, not redeploying just making a hot change to the jsp. * Geoff will be rebooting qball next week to reconfigure it with more mem and cpus. (Qball is a virtual machine so this is a change to the xen vm). --- Mon Oct 12 16:50:55 PDT 2009 * Changing process_worker_run to set java opts. Not redeploying - changing in place. Will get clobberred by next deployment. Temporary fix until Geoff reconfigures the machine --- Mon Oct 19 13:07:38 PDT 2009 * Geoff reconfigured with 2 cpus, same amount of mem. I'm redeploying R5. --- Fri Oct 23 09:43:26 PDT 2009 Terri Not redeploying but setting up some things so we can use globus from this account. I want to install the cron job that cleans up working dirs on teragrid hosts. At the same time I'll do some of the preliminary stuff needed for deploying the globus enabled application here, like adding stuff to ```~/.m2/settings.xml```. * installed cron job to remove old workspace dirs on abe and lonestar. (see crontab -l). Script it runs is ```~/scripts/tg_ws_cleanup.sh```. * installed cron job to update ```~/.globus/certifcates``` on first of month. Script is ```~/scripts/update_crl.sh```. --- ### Deployment instructions To deploy a new version: - Log into ```portal2test@8ball.sdsc.edu``` * ```cd cipres-portal-src``` and use the ```release.sh``` script to create a branch and check it out. * ```cd cipres-portal-src//cipres-portal```, ```cp scripts/tl.sh```, modify ```-Denv=argument``` in ```tl.sh```. * use ```tl.sh``` to build and deploy. * when finished testing on 8ball, use ```release.sh``` to create a tag. - Log into ```portal2prod@rhel50x86-testing.sdsc.edu``` * ```cd cipres-portal-src``` and use ```release.sh``` to check out the tag. * ```cd cipres-portal-src//cipres-portal```, ```cp scripts/tl.sh```, modify ```-Denv=argument``` in ```tl.sh```. * use ```tl.sh``` to build and deploy. * ```cp target/portal2.war``` to ```cipres-portal-src//cipres-portal```, add it to svn and commit the change. Minimum tests after new deployment: 1) Run one sge job and one non-sge job (cipres-framework), eg. clustalw and paup. 2) Find sdk.log entries for both jobs, find process worker entries for each in logs/running_jobs, or logs/finished_jobs. 3) Check results via web application and check for jobs' working directories under ARCHIVED. 4) Make sure you get a job complete email for each job. I (Terri) removed anti-resourceLocking and anti-jarLocking from the web server so YOU MUST RESTART tomcat when you redeploy. --- ### TroubleShooting Old cipres framework jobs, like "paup": These are run by ssh'ing into snooker (see tool.resource.host in cipres-prod.properties) and running the tool /projects/ngbw_db1/contrib/tools/cipreswrapper/v1.0/run_cipres_svc.sh. Look here to see which cipres cluster node must be running a registry (currently cipres12). A registry must also be running on snooker. To check the registry on snooker, log in as bwbatch@snooker and make sure cipres registry and prop svc are running: Look for them: ```ps -U bwbatch -f -ww |grep java``` and restart them with ```~/cipres_stuff/cipres_bwbatch```. To check the registry on the cluster, log into the node specified in ```run_cipres_svc.sh``` as user "cipresd", ```cd``` to ```~/bin``` and run ```cipresd`. See also ```~/Readme.txt```. If you try to restart tomcat and see "Bind address already in use" exceptions in ```$CATALINA_HOME/logs/catalina.out```, it may be that tomcat isn't being killed. ```netstat -a -p | grep 7070``` will show you which processes are still using the tomcat port 7070. You'll probably see them in CLOSE_WAIT state. Do ```ps aux | grep pid``` to verify that it's tomcat and then kill the process. Now you can do tomcat_start.