Proactive Federation Management with ELK for Enhanced Performance
"Learn about proactive management of federation using ELK (Elasticsearch, Logstash, Kibana) for improved performance and real-time auditing. Explore how HEAnet leverages this system to generate yearly stats on Edugate usage and identify popular services. Discover the transition from legacy systems to the new stats system for efficient data visualization. Join us on May 28, 2025, for a demo and Q&A session."
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Proactive Management of Federation using ELK Yasvanth Babu ELK Elasticsearch, Logstash, Kibana HEAnet
About me About me System administrator at HEAnet Team - Edugate Interest... Explore new things. May 28, 2025 2
Agenda Agenda Aim, Problem statement. Edugate and Elasticsearch. Outcomes. Demo. Q&A. May 28, 2025 3
Aim Aim To Generate yearly stats on Edugate Usage. Increasing or decreasing. To find popular services in Edugate and who makes better use of Edugate. May 28, 2025 4
Legacy system Legacy system Open-source. Build at Cardiff University. Used mainly to parse authentication events. IDP, OpenAthens and Ezproxy. HEAnet used raptor to generate IDP audit statistic. May 28, 2025 5
Raptor Architecture. Raptor Architecture. May 28, 2025 6
Stats Stats May 28, 2025 7
Problem Problem Raptor code was not actively maintained. New code base will be released soon. rrdtool graphs not interactive. Lack in self-service and real-time auditing. Very slow performance. May 28, 2025 8
Edugate Edugate And Elasticsearch And Elasticsearch A graph/stats system built on top of Elastic Stack. Elasticsearch. Distributed Search engine. All data s are indexed. Logstash. Pipeline processing. Kibana UI, Visualize your data. Charts, Maps, Time-series May 28, 2025 9
New stats system New stats system May 28, 2025 10
May 28, 2025 11
Achieved... Achieved... To Generate yearly stats on Edugate Usage. To find popular services in Edugate and who makes better use of Edugate. May 28, 2025 12
Addressed the problems that aren't noticed Addressed the problems that aren't noticed for months for months May 28, 2025 13
Additional outcomes... Additional outcomes... Edugate Access Audits: No attributes released. Important journals. Ligo, Clarin, Niche-resources. Suspicious Logins. Compromised Accounts. Misconfigured SP. Sharing of credentials. Service Activity. Login predictions. Real-time alerts. May 28, 2025 14
What data? What data? 52.51.67.68 ------------------------------------------------------------------------------------------------------------------>>> IP address Idp1.heanet.ie ---------------------------------------------------------------------------------------------------------------->>> Host 20180607T232814Z --------------------------------------------------------------------------------------------------------->>> Timestamp urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect -------------------------------------------------------------->>> Request Binding _85b888a75ac583060cf1489768d20a15 ----------------------------------------------------------------------------->>> Request ID https://edugate.heanet.ie/shibboleth ------------------------------------------------------------------------------->>> Relying Party ID http://shibboleth.net/ns/profiles/saml2/sso/browser ------------------------------------------------------------>>> Message Profile ID https://idp.heanet.ie/idp/shibboleth ----------------------------------------------------------------------------------->>> Asserting Party ID urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST ----------------------------------------------------------------->>> Response Binding _8c804d811a89b8fbfb79b462bf65f76e ----------------------------------------------------------------------------->>> Response ID Joe.bloggs -------------------------------------------------------------------------------------------------------------------->>> User Name urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport ---------------------------------------->>> AuthMethod eduPersonPrincipalName,email ---------------------------------------------------------------------------------------->>> Released Attributes AAdzyuWqsRuNM5ySZ2I724XTrL2/MzmvJU= ------------------------------------------------------------------------>>> Name Identifier _79ff78ddfba9a21d38401c06e19e163f ------------------------------------------------------------------------------->>> Assertion ID May 28, 2025 15
Self Self- -service service Kibana lacks in Multi-tenant. Security Anonymize usernames. Shard access. May 28, 2025 16
Searchguard Searchguard Security Add-on for ELK Stack. Encryption. Authentication. LDAP/AD, Kerberos, Host-Based, JWT and SAML . Multi-tenancy. Based on user roles. Centralized User ACL Index. Licensing. Community, Academic and Enterprise May 28, 2025 17
Future work Future work Learning analytics. IDP, Eduroam and VLE ( moodle). Automation. Resource registry. Can Integrate with: Netflow, Application data, Web-filter data. Machine learning. May 28, 2025 18
Summary... Summary... Multi-tenant Stats system. Real-Time Auditing and Analyzing. Alerting System. May 28, 2025 19
Demo... Demo... May 28, 2025 20
Questions ? Questions ? May 28, 2025 21
