Edgio AppOpsEdgio DeliveryEdgio StreamingContact SalesContact SupportResourcesInvestorsCareersDeliveryApp EdgeEdgeJS (CDN-as-code)ObservabilityTraffic SplittingGlobal CDNFull-Stack EnvironmentsFeature ManagementBranch PreviewPredictive PrefetchServerless ComputeGraphQL CachingImage OptimizationApp PlatformWeb Application FrameworksIterative MigrationApp SecurityWAF, DDoS, Bot Management and API ProtectionDevelop fasterRelease with confidenceRun sub-second sitesProtect your appSimplify, save and evolveSecurityStreamingExpert ServicesEdge Cache for ISPsAccelerated DevelopmentInstant Page LoadsNext-Gen Web CDNGlobal CDNWeb SecurityDocsContact SalesApp EdgeApp PlatformApp SecurityBranch PreviewEdgeJSFeature ManagementFull-stack EnvironmentsGlobal CDNGraphQL CachingImage OptimizationIterative MigrationObservabilityPredictive PrefetchServerless ComputeTraffic SplittingWAF, DDoS, Bot ManagementWeb Application FrameworksDeliveryStreamingSupportResource CenterBlogJavaScript JamDeveloper DocsChangelogForumAboutLeadership TeamCareersInvestorsNewsroomContact UsTerms of ServicePrivacy PolicyAcceptable Use PolicyData and Protection AddendumInvoice MethodologiesPrivacy ShieldWebsite Disclaimer
Blog

Real-Time Log Delivery Now Supports Both CDN and Security Logs

May 13, 2021
Print Article

Original source: Edgecast

Our customers rely on Real-Time Log Delivery (RTLD) to obtain critical insights on the performance of our delivery network, identifying issues and making changes when needed. We are excited to announce that we have launched a new feature of our Real-Time Log Delivery that enables customers to stream WAF events into RTLD. Our RTLD WAF delivers log data for threats identified by our WAF's access and managed rules. It currently excludes log data for rate-limited requests, but look for this functionality supported in an upcoming release.

In addition to our new security feature, we have made several other updates to RTLD to help our customers collect, process, and analyze the data they need to generate the exact visualizations to help them make better-informed performance decisions.  Here is a summary of the new functionality:

  • Endpoints: Customers can stream raw CDN performance and security logs in less than 60 seconds to different supported endpoints, including Amazon S3, Splunk, Sumologic, Azure Blob Storage, Datadog, or their own web server via HTTP Post. Upcoming releases will include support for New Relic and Google Cloud Storage.

  • Profiles: RTLD offers broad customization with the ability to create multiple profiles. Each profile operates independently and allows full flexibility to select destination, filtering, and output fields per profile. Since each profile is independent, more than one profile may be configured to deliver the same log data set.

  • Filtering: Filtering allows our customers to use RTLD for only specific segments of their overall traffic profiles, which helps reduce costs and resources. By focusing on the data that matters most to them, our customers can make better decisions sooner. Customers can filter CDN logs by edge CNAME, HTTP response codes (2xx, 3xx, 4xx, or 5xx), or user agent.  RTLD WAF logs can be filtered by edge CNAME, country code, security application manager, access rule, custom rule, or managed rule configuration.

  • Downsampling: In addition to filtering, customers also have the option for downsampling logs to 0.1%, 1%, 25%, 50%, or 75% of the set of log entries that will be delivered. For example, downsampling 1 million log entries to 1% would result in 10,000 log entries.

  • APIs: Customers can configure RTLD via APIs to gain efficiencies and interface our network with their preferred configuration method. RTLD configuration APIs give customers an efficient method for setting up and changing RTLD settings. See the architecture diagram below for details.  Access our REST API help center for RTLD here

  • Fields: RTLD CDN and RTLD WAF offer several fields to choose from. The fields are categorized into related buckets, making it easy for customers to select the information they want to receive while avoiding data not relevant to their business needs. With RTLD CDN, any info in a custom request or response header can be added as a data field to report on information being captured in headers. Cookie values can also be selected. A complete list of CDN log fields can be found here. View WAF log fields here.

  • Log Formats: For HTTP posts, AWS S3, and Azure Blob Storage integrations, customers can choose from JSON, JSON Array, and JSON Lines formats. We will be adding .csv support in an upcoming release. ‍

RTLD ARCHITECTURE

RTLD supports several use cases, including:

Real-time CDN log monitoring

  • For general delivery of the logs, CDN administrators can monitor in real time how traffic is delivered and catch any performance issues or errors as they occur.

Large live events monitoring

  • Live sports use case: The Verizon Media Platform supports the streaming and delivering multiple live sports events. Using Real-Time Log Delivery in conjunction with video solutions, like slicer monitoring and ad data, provides an excellent way to monitor these events and act quickly should issues arise.

Business intelligence

  • Integrate with third-party log delivery endpoints to highlight traffic insights and trends to enable informed decision-making.

  • Multiple profile use case: Send full log data to S3 for batch processing while a separate profile sends non-2xx responses to Splunk to alert you of problems.

Get the intelligence you need to manage your web application and infrastructure in real time. Make better decisions faster with RTLD.

Contact us today to learn more about this update and RTLD.

Explore Edgio Solutions

Get the information you need. When you’re ready, chat with us, get an assessment or start your free trial.