Publications
Our published Work!
2024
Shi, Wei-Pei; Nordling, Torbjörn E M
Combining old school autoencoder with Cotracker for improved skin feature tracking Proceedings Article
In: The 19th IEEE Conference on Industrial Electronics and Applications (ICIEA 2024), IEEE, Kristiansand, Norway, 2024.
Abstract | BibTeX | Tags: autoencoder, Convolutional neural network, Cotracker, Deep feature encoder, human motion assessment, Skin feature tracking, Transformer
@inproceedings{Shi2024ICIEA,
title = {Combining old school autoencoder with Cotracker for improved skin feature tracking},
author = {Wei-Pei Shi and Torbjörn E M Nordling},
year = {2024},
date = {2024-08-01},
urldate = {2024-08-01},
booktitle = {The 19th IEEE Conference on Industrial Electronics and Applications (ICIEA 2024)},
publisher = {IEEE},
address = {Kristiansand, Norway},
series = {IEEE Conference on Industrial Electronics and Applications (ICIEA 2024)},
abstract = {Abstract—Background: Skin feature tracking enables quantification of human
motion in an explainable way, making it suitable for clinical assessments.
Accuracy is crucial, but no study has investigated state-of-the-art deep neural
network-based point tracking models such as Cotracker. Cotracker jointly tracks
points and has been shown to have better 3-pixel accuracy than five other
state-of-the-art deep learning methods on the two most commonly used datasets
for evaluation of single target point tracking. In 2021, Chang and Nordling
introduced the Deep Feature Encoder (DFE) and demonstrated skin feature tracking
so accurate that the errors cannot be excluded to stem from the manual labeling
of the videos based on a χ2-test.
Problem: How accurately can different methods track skin features and how to avoid
the intrinsic weaknesses of the methods?
Methods: We use videos of the Unified Parkinson’s Disease Rating Scale postural
tremor test recorded at two hospitals for benchmarking. DFE utilizes the encoder
part of an autoencoder consisting of a five-layer convolutional neural network
trained to reproduce skin crops without supervision. The residual squared error
of the latent features of the encoder is then compared with crops to obtain a
predicted position. We also propose Cotracker DFE, using Cotracker to obtain an
approximate position and subsequently cropping a small area that is fed to DFE
to obtain a position predicted with a lower mean pixel error.
Results: The mean Euclidean distance errors of Cotracker, DFE, and Cotracker-DFE
are 1.2, 0.8, and 0.8 pixels, respectively. DFE requires time-consuming computations,
making it 35 times slower than Cotracker.
Conclusion: The old school DFE provided more accurate skin feature tracking, while
combining DFE with Cotracker provides the best overall performance, circumventing
the lack of labeled data and computational resources required to fine-tune Cotracker.},
howpublished = {The 19th IEEE Conference on Industrial Electronics and Applications (ICIEA 2024), in Kristiansand, Norway 05-08 August 2024},
keywords = {autoencoder, Convolutional neural network, Cotracker, Deep feature encoder, human motion assessment, Skin feature tracking, Transformer},
pubstate = {published},
tppubtype = {inproceedings}
}
Abstract—Background: Skin feature tracking enables quantification of human
motion in an explainable way, making it suitable for clinical assessments.
Accuracy is crucial, but no study has investigated state-of-the-art deep neural
network-based point tracking models such as Cotracker. Cotracker jointly tracks
points and has been shown to have better 3-pixel accuracy than five other
state-of-the-art deep learning methods on the two most commonly used datasets
for evaluation of single target point tracking. In 2021, Chang and Nordling
introduced the Deep Feature Encoder (DFE) and demonstrated skin feature tracking
so accurate that the errors cannot be excluded to stem from the manual labeling
of the videos based on a χ2-test.
Problem: How accurately can different methods track skin features and how to avoid
the intrinsic weaknesses of the methods?
Methods: We use videos of the Unified Parkinson’s Disease Rating Scale postural
tremor test recorded at two hospitals for benchmarking. DFE utilizes the encoder
part of an autoencoder consisting of a five-layer convolutional neural network
trained to reproduce skin crops without supervision. The residual squared error
of the latent features of the encoder is then compared with crops to obtain a
predicted position. We also propose Cotracker DFE, using Cotracker to obtain an
approximate position and subsequently cropping a small area that is fed to DFE
to obtain a position predicted with a lower mean pixel error.
Results: The mean Euclidean distance errors of Cotracker, DFE, and Cotracker-DFE
are 1.2, 0.8, and 0.8 pixels, respectively. DFE requires time-consuming computations,
making it 35 times slower than Cotracker.
Conclusion: The old school DFE provided more accurate skin feature tracking, while
combining DFE with Cotracker provides the best overall performance, circumventing
the lack of labeled data and computational resources required to fine-tune Cotracker.
motion in an explainable way, making it suitable for clinical assessments.
Accuracy is crucial, but no study has investigated state-of-the-art deep neural
network-based point tracking models such as Cotracker. Cotracker jointly tracks
points and has been shown to have better 3-pixel accuracy than five other
state-of-the-art deep learning methods on the two most commonly used datasets
for evaluation of single target point tracking. In 2021, Chang and Nordling
introduced the Deep Feature Encoder (DFE) and demonstrated skin feature tracking
so accurate that the errors cannot be excluded to stem from the manual labeling
of the videos based on a χ2-test.
Problem: How accurately can different methods track skin features and how to avoid
the intrinsic weaknesses of the methods?
Methods: We use videos of the Unified Parkinson’s Disease Rating Scale postural
tremor test recorded at two hospitals for benchmarking. DFE utilizes the encoder
part of an autoencoder consisting of a five-layer convolutional neural network
trained to reproduce skin crops without supervision. The residual squared error
of the latent features of the encoder is then compared with crops to obtain a
predicted position. We also propose Cotracker DFE, using Cotracker to obtain an
approximate position and subsequently cropping a small area that is fed to DFE
to obtain a position predicted with a lower mean pixel error.
Results: The mean Euclidean distance errors of Cotracker, DFE, and Cotracker-DFE
are 1.2, 0.8, and 0.8 pixels, respectively. DFE requires time-consuming computations,
making it 35 times slower than Cotracker.
Conclusion: The old school DFE provided more accurate skin feature tracking, while
combining DFE with Cotracker provides the best overall performance, circumventing
the lack of labeled data and computational resources required to fine-tune Cotracker.
2019
Lin, Chia-Ming; Tsai, Po-Jung; Nordling, Torbjörn E M
ProtFunAI–An artificial intelligence for prediction of protein function from protein sequence alone Miscellaneous
The 20th International Conference on Systems Biology (ICSB-2019) in Okinawa Institute of Science and Technology Graduate University, Okinawa, Japan, 2019.
Abstract | Links | BibTeX | Tags: Convolutional neural network, Protein function, Systems Biology
@misc{Nordling2019ICSB,
title = {ProtFunAI–An artificial intelligence for prediction of protein function from protein sequence alone},
author = {Chia-Ming Lin and Po-Jung Tsai and Torbjörn E M Nordling},
url = {https://www2.aeplan.co.jp/icsb2019/images/Program191009.pdf},
year = {2019},
date = {2019-11-01},
booktitle = {The 20th International Conference on Systems Biology (ICSB-2019) in Okinawa Institute of Science and Technology Graduate University, Okinawa, Japan: Abstract book},
pages = {52},
publisher = {Okinawa Institute of Science and Technology (OIST) Graduate University},
address = {Okinawa Institute of Science and Technology Graduate University, Okinawa, Japan},
abstract = {Knowledge about the function of a protein is essential for understanding its role. Prediction of protein functions from the protein sequence alone using computational methods has been attempted. Previously, we created FFANEprot— a deep convolutional neural network trained on a dataset of 81,267 proteins and 1,169 Gene Ontology (GO) terms of the molecular function (MF) from the Swiss-Prot database. It is the best predictor of GO MFs from protein sequence alone, with training and test Matthews correlation coefficients (accuracies) of 0.52 (98.84%) and 0.49 (98.67%), respectively. Based on FFANEprot, we here present the ProtFunAI web service (protfunai.nordlinglab.org) consisting of a database of MF predictions of 20,405 reviewed human proteins and a prediction service that can predict the MF of any supplied protein sequence within roughly a minute.},
howpublished = {The 20th International Conference on Systems Biology (ICSB-2019) in Okinawa Institute of Science and Technology Graduate University, Okinawa, Japan},
keywords = {Convolutional neural network, Protein function, Systems Biology},
pubstate = {published},
tppubtype = {misc}
}
Knowledge about the function of a protein is essential for understanding its role. Prediction of protein functions from the protein sequence alone using computational methods has been attempted. Previously, we created FFANEprot— a deep convolutional neural network trained on a dataset of 81,267 proteins and 1,169 Gene Ontology (GO) terms of the molecular function (MF) from the Swiss-Prot database. It is the best predictor of GO MFs from protein sequence alone, with training and test Matthews correlation coefficients (accuracies) of 0.52 (98.84%) and 0.49 (98.67%), respectively. Based on FFANEprot, we here present the ProtFunAI web service (protfunai.nordlinglab.org) consisting of a database of MF predictions of 20,405 reviewed human proteins and a prediction service that can predict the MF of any supplied protein sequence within roughly a minute.
2018
Liou, Yi-Fan; Tsai, Po-Jung; Huang, Zi-Yu; Chiou, Po-Chin; Chu, Hsiao-Wei; Ciou, Li-Ping; Nordling, Torbjörn E. M.
FFANEprot: Predicting Protein Functions using a Weight-sharing Multitask Neural Network Optimized by a Firefly Algorithm with Natural Enemy Strategy Proceedings Article
In: 17th International Conference on Bioinformatics (INCoB-2018), Asia Pacific Bioinformatics Network (APBioNet), New Delhi, India, 2018.
Abstract | Links | BibTeX | Tags: Convolutional neural network, deep learning, Evolutionary algorithm, Firefly algorithm, Inhibitory neurons, Natural enemy strategy
@inproceedings{Liou2018,
title = {FFANEprot: Predicting Protein Functions using a Weight-sharing Multitask Neural Network Optimized by a Firefly Algorithm with Natural Enemy Strategy},
author = {Yi-Fan Liou and Po-Jung Tsai and Zi-Yu Huang and Po-Chin Chiou and Hsiao-Wei Chu and Li-Ping Ciou and Torbjörn E. M. Nordling},
url = {http://www.incob2018.org/},
year = {2018},
date = {2018-09-01},
booktitle = {17th International Conference on Bioinformatics (INCoB-2018)},
publisher = {Asia Pacific Bioinformatics Network (APBioNet)},
address = {New Delhi, India},
abstract = {Background: The prediction of multiple functions of several proteins at once from the protein sequence alone is essential, but difficult. To solve this problem, we composed a dataset of 81,267 proteins and 1,169 Gene Ontology (GO) terms of the molecular function (MF) from the Swiss-Prot database, and used weight-sharing and multi-task learning to create FFANEprot. Results: The architecture of FFANEprot was optimised by a Firefly algorithm with a natural enemy strategy (FFANE), i.e. periodic reversals. The training and test Matthews correlation coefficients (accuracies) are 0.52 (98.84%) and 0.49 (98.67%), respectively. When analysing the trained networks, we found many completely inhibitory neurons, which typically have a small kernel size and occupy approximately 30% of the CNNs. Conclusion: FFANEprot can predict GO MF terms with high accuracy from sequence alone. Our FFANEnet source code is available at http://ffanenet.nordlinglab.org.},
keywords = {Convolutional neural network, deep learning, Evolutionary algorithm, Firefly algorithm, Inhibitory neurons, Natural enemy strategy},
pubstate = {published},
tppubtype = {inproceedings}
}
Background: The prediction of multiple functions of several proteins at once from the protein sequence alone is essential, but difficult. To solve this problem, we composed a dataset of 81,267 proteins and 1,169 Gene Ontology (GO) terms of the molecular function (MF) from the Swiss-Prot database, and used weight-sharing and multi-task learning to create FFANEprot. Results: The architecture of FFANEprot was optimised by a Firefly algorithm with a natural enemy strategy (FFANE), i.e. periodic reversals. The training and test Matthews correlation coefficients (accuracies) are 0.52 (98.84%) and 0.49 (98.67%), respectively. When analysing the trained networks, we found many completely inhibitory neurons, which typically have a small kernel size and occupy approximately 30% of the CNNs. Conclusion: FFANEprot can predict GO MF terms with high accuracy from sequence alone. Our FFANEnet source code is available at http://ffanenet.nordlinglab.org.
Subscribe To Our Newsletter
Nordling Lab News will provide you with irregular updates on our research and topics of interest.