GitLab

Fixed bug; When importing, LOVD could have used the wrong data for checking the panelid, fatherid and motherid links.
- Links to records in the file should take preference over links with database entries, instead of the other way around.
- Links to the own ID aren't allowed, so stop throwing additional, confusing, error messages.
- The panel size was checked only when the Individual/Gender column was active.
- Also, checking for the Individual/Gender column happened once or twice per data row, instead of once per import.

Several fixes in the security of ViewLists.

- Allow Curators to download the gene-specific Individuals view.
- Now that we have code authorizing Curators for VLs on the Individual's VE, let Curators download the Screening and Variant VLs there.
- Now that we have code authorizing Curators for the Variants VL on the Screenings VE, let Curators download it.

- This means Curators couldn't search for non-public Screenings on the VL on this VE.
- Now it loads the needed authorization.

Fixed bug; Variant ViewLists on Individual and Screening VEs were authorized for Curators, but the Ajax-VL wasn't.
- This means Curators can't search for non-public variants on the VL on either of these VEs.
- Now it loads the needed authorization.

- Hiding columns for a VL is stored in session upon defining the VL. However, VLs can be loaded independently of being defined first, for the External Viewer.
- To allow for the External Viewer to hide columns, VLs could be told to hide additional columns. However, this was overwriting the current list of hidden columns, allowing special requests that would columns previously hidden.
- For the Users VL, which was opened up to Submitters due to the Colleagues feature, overwriting the ColsToSkip was prevented in the VL code specifically, to hide sensitive data.
- Added a global solution that makes sure for other VLs the list doesn't get overwritten either, but merged instead. This prevents pre-defined VLs to lose their ColsToSkip using specially crafted requests.
- However, as the External Viewer just loads undefined VLs, it can be configured to show all columns in these VLs. As such, if the ColsToSkip feature is ever used to hide sensitive data, these columns need to be defined in the VL code itself.

- During Find & Replace implementation, the code was changed to load the gene-specific authorization using $_REQUEST rather than $_GET. However, the filtering of the results was still using $_GET. Hence, authorizations can be loaded forging a $_POST request while not necessarily filtering the results using $_GET.
- This allowed Curators to craft special requests to load VLs with non-public data of other genes.
- Solved this by enforcing filtering when authorizing Curators.

Some small fixes to the update checker.

- Changed the icon for "No updates available" to a simple green mark.
- Added Alt and Title tags to the image, so users can understand what it means without having to click it.

Fix annoying error that we have a problem checking for updates if we're simply running an updated version.
- The update checker reports a version conflict if the local version is higher than the version online, but we don't want to release every single subversion (like 3.0-21a, 3.0-21b, 3.0-21c, etc).
- Stop complaining if our root is up to date, but we have a subversion higher.

Submission API improvements

Updated the LOVD2 to LOVD3 converter.

Fixed a minor data formatting bug in HPO terms and added some automated data formatting for RNA and protein fields.
- HPO terms should not be stored as "HP:123456 (term)" but instead as "term (HP:123456)".
- If RNA has not been checked, the RNA description will automatically get parentheses if they are absent.
- If RNA has not been checked, the protein description will automatically get parentheses if they are absent.
- The default protein change is now only "-" for non-coding transcripts. For coding transcripts, it's now "p.(?)".

Added support for converting phenotype OMIM IDs to LOVD3 Diseases, and support for the non-official Individual/Fam_Pat column.

- Added mappings for Patient/Age_death, Patient/Age_onset and Patient/Age_diagnosis.
- Updated value conversion for gender, geographic origin, screening technique, screening template and genetic origin.

- Can now also handle DOIs.
- Added support for {PMID:PubMed 2019 abstract:12345678} format. The script will fetch the paper's author and year using eutils and rewrite the reference to {PMID:Author et al (Year):12345678}.
- Implemented a cache to make sure eutils doesn't get the same question twice.
- Updated lovd_callJSONService() to be able to return an array instead of an object.

Reverted ONLY_FULL_GROUP_BY changes.

- It makes no sense to order on something, if we do a GROUP BY on the PK that we also do a WHERE on. Only one line will be returned anyway.
- Added sorting to the GROUP_CONCAT() so that the gene symbols will be sorted in the email's subject.
- Don't overcomplicate the query for fetching curators.

- Don't pull out extra fields out of the database, just because we're sorting on it.
- Don't sort on a field alias if you can sort on the actual fields.

- Genes are assumed to be in sense when at least one transcript is in sense.

- Also simplified SELECTs whereever possible.
- Custom VLs with VOG as the first table (Variant VL on Screenings VE and Individual VE) no longer show the minimal VOT's effectid, but instead the VOG's effectid, which makes much more sense.
- These changes might also lead to some speedups.

- Reviewed all MIN()/MAX() uses.
- For BED files, always return the mapping on the earliest transcript.
- For the genes list, assume the gene is in sense when at least one transcript is in sense.

Fix `variants/in_gene` view.

- Removed more min() and max() calls.

- This speeds up the query quite a lot.
- Also, rename LEFT JOINs to LEFT OUTER JOINs to make more clear what they are.

- Even if we try to play nice, the totally unnecessary MIN() and MAX() calls slow down queries a lot.
- See #386.

Fixed some problems with the DOI view.

- Pagination was off, but especially if you're searching by journal you need pagination.
- Fixed the journal search; DOIs can also contain slashes and journal names can contain dots.
- Cleaned up the code a bit.

Allow for publisher and journal-specific queries using the DOI reference search.

Enlarge the DNA and protein fields to support longer insertions.

- Normally DOI reference searches are on the full DOI only (like 10.1002/humu.21438). Now, also allow for searches using the journal's DOI, like 10.1002/humu.
- Implements #257.

- This allows for larger insertions to be stored.
- LOVD+ is using the same size, so this also removes one difference between LOVD3 and LOVD+.

- This allows for larger insertions to be stored.
- LOVD+ is using the same sizes, so this also reduces one difference between LOVD3 and LOVD+.

Multiple bug fixes.

Updated the ReadingFrameChecker; it now provides the correct (new) HGVS nomenclature.