Understanding OpenTelemetry for Ruby on Rails

Observability has evolved from a simple monitoring practice into a critical system design principle that helps development teams build more reliable, maintainable applications. In this comprehensive guide, we explore how OpenTelemetry is transforming observability for Ruby on Rails developers, based on insights from Kayla Riopelo, a lead software engineer at New Relic who works on both the Ruby agent and OpenTelemetry Ruby gems.

What Is Observability and Why Does It Matter?

Observability goes beyond traditional monitoring. While monitoring helps you answer questions you already know to ask (known unknowns), observability enables you to investigate issues you didn’t anticipate (unknown unknowns). In practical terms, observability means having the ability to understand your system’s internal state by examining its outputs.

When you deploy a Rails application into production, users interact with it in ways you might not expect. Observability captures this information and creates an archive that helps you understand what’s happening inside your system without needing to predict every possible scenario in advance.

Signs Your Rails Application Needs Better Observability

Several indicators suggest your system could benefit from improved observability:

• Abandoned shopping carts due to performance issues
• Slow request times affecting user experience
• Support tickets reporting features that aren’t working
• Team members struggling to understand or debug recent features
• Uncertainty about whether deployed features are being used
• Difficulty connecting system performance to business outcomes

What Is OpenTelemetry?

OpenTelemetry represents a fundamental shift in how we approach application instrumentation. Rather than installing vendor-specific gems like New Relic, DataDog, or AppSignal directly into your application, OpenTelemetry provides a vendor-agnostic alternative.

Think of it as similar to payment processing: just as you might want to support both Stripe and PayPal without tightly coupling your code to either vendor, OpenTelemetry allows you to collect telemetry data using open-source tools and send that data to multiple observability platforms simultaneously.

The Three Pillars of OpenTelemetry

OpenTelemetry organizes observability data into three primary types, called signals:

1. Traces Traces tell the story of a request as it flows through your system. Each trace consists of individual pieces called spans. In a typical Rails request, you might see spans for:

• The controller action
• Database queries through Active Record
• View rendering
• External API calls
• Individual Rack middleware calls

2. Metrics Metrics provide aggregated time series data about your system’s performance. Common examples include request duration, active job counts, and queue sizes. OpenTelemetry supports several metric types (called instruments):

• Counters for values that only increase
• UpDownCounters for values that can increase or decrease
• Gauges for point-in-time snapshots
• Histograms for value distributions

3. Logs Logs are the traditional form of observability—timestamped text records of events. OpenTelemetry enhances logs by placing them in context with traces and spans, making it easier to understand what was happening in your system when a particular log entry was created.

Current State of OpenTelemetry in Ruby

The Ruby implementation of OpenTelemetry has reached different stability levels for each signal:

• Traces: Marked as stable, with full specification compliance
• Metrics: Experimental but functional for basic use cases like counters, histograms, and gauges
• Logs: Experimental but quite stable in practice, with most features implemented

The experimental designation doesn’t mean these features are unreliable. Rather, it indicates they need real-world usage and testing before the technical committee can mark them as fully stable. The Ruby community needs developers to adopt and test these tools in production environments.

Getting Started with OpenTelemetry in Rails

Basic Setup

Setting up OpenTelemetry in a Rails application requires more initial configuration than vendor-specific gems, but provides greater flexibility:

1. Add the necessary gems to your Gemfile
2. Create an initializer that calls OpenTelemetry::SDK.configure
3. Specify which instrumentations to enable using use or use_all
4. Configure your exporter through environment variables

The use_all method automatically enables all compatible instrumentations found in your system, making it easy to get comprehensive coverage quickly. If you prefer expert guidance, RoR Wizards offers Rails consulting for performance instrumentation and observability to help you design and deploy OpenTelemetry correctly.

Active Support Notifications Integration

One of OpenTelemetry’s most powerful features for Rails developers is its integration with Active Support Notifications. This publish-subscribe interface powers much of Rails’ internal instrumentation and serves as the foundation for most observability tools.

OpenTelemetry includes an Active Support instrumentation gem that can automatically convert your custom notifications into spans. This means you can instrument your code using familiar Rails patterns without adding OpenTelemetry-specific APIs everywhere.

For code that falls outside standard Rails conventions—like service objects or custom business logic—you can create your own Active Support notifications and have them automatically instrumented by OpenTelemetry.

Practical Applications for Rails Developers

Monitoring Background Jobs

Background jobs represent a common area where additional visibility proves valuable. Here’s a practical example using metrics:

Create an UpDownCounter to track active jobs. When a job starts, increment the counter by one. When it completes, decrement by one. This gives you real-time visibility into how many jobs are running at any moment, helping you:

• Identify capacity issues before they become problems
• Understand usage patterns throughout the day
• Make informed decisions about worker scaling
• Correlate business events with system load

You can enhance this further by adding dimensions to your metrics, such as queue name or job class, allowing you to analyze the data from multiple perspectives.

Performance Optimization in Development

Observability isn’t just for production environments. During local development, you can use OpenTelemetry to compare different implementation approaches. When you have multiple ways to solve a problem and aren’t sure which will perform better, add spans around each approach and measure the differences in your development environment.

This technique complements traditional benchmarking by providing context about how different approaches behave within your actual application stack.

Enhanced Logging with Context

The recently released logger bridge (opentelemetry-instrumentation-logger) captures all logs created through Ruby’s standard logger and enriches them with OpenTelemetry context. This means your existing logs automatically gain correlation with traces and spans, making debugging significantly easier.

When investigating an issue, you can see not just the log messages, but also the exact request context, timing information, and related spans—all without changing your existing logging code.

Working with the OpenTelemetry Collector

The OpenTelemetry Collector is a separate executable that processes telemetry data before sending it to backends. It offers extensive customization through plugins and extensions, allowing you to:

• Filter data before it leaves your infrastructure
• Add or modify attributes
• Sample traces based on custom rules
• Route different data types to different backends
• Transform data to meet specific requirements

This processing layer gives you fine-grained control over what data gets stored and where, potentially reducing costs while maintaining the observability you need.

Cost Management Strategies

As applications scale, telemetry costs can become significant. OpenTelemetry provides several strategies for managing these costs:

Selective Instrumentation

Use Ruby conditionals to instrument only what matters. For example, you might collect detailed traces for paying customers while using lighter instrumentation for free-tier users.

Span Processors

Create custom span processors that can modify or drop spans before they’re exported. This allows you to collect comprehensive data initially, then selectively remove unnecessary information based on attributes like user tier or feature flags.

Feature Flags Integration

Many teams use feature flags to control observability without redeploying code. This enables you to quickly enable detailed instrumentation when investigating issues, then disable it once resolved.

Data Storage and Retention

OpenTelemetry handles collection and standardization, but storage duration is managed by your chosen backend. This separation allows you to optimize costs by selecting appropriate retention periods for different data types.

If you want a turnkey implementation, many teams rely on RoR Wizards’ DevOps services with observability to set up robust tracking, logging, and metrics pipelines.

The Vendor-Neutral Advantage

OpenTelemetry’s vendor neutrality creates interesting competitive dynamics. Previously, observability vendors differentiated themselves primarily through the data they collected. OpenTelemetry challenges this by providing standardized, high-quality instrumentation that’s openly available.

Now vendors compete on:

• Pricing models and affordability
• Data visualization and user experience
• Query capabilities and analysis tools
• Integration ecosystems
• Support and documentation

This shift benefits developers by reducing vendor lock-in and making it easier to evaluate different platforms or use multiple vendors simultaneously.

Testing Your Observability

While not common, testing instrumentation code provides valuable confidence, especially for custom instrumentation. OpenTelemetry supports this through special exporters:

• Console Exporter: Outputs telemetry data to the console for manual inspection
• In-Memory Exporter: Captures data in memory for programmatic assertions in tests

These tools enable you to verify that your instrumentation creates the expected spans, metrics, and logs without sending data to external services.

OpenTelemetry for CI and Testing

Beyond production observability, OpenTelemetry can instrument your test suite itself. This provides insights into:

• Frequently failing tests that need attention
• Test suite performance bottlenecks
• CI costs and resource usage
• Test execution patterns over time

Open Source Alternatives to Commercial Vendors

You don’t need a commercial observability platform to use OpenTelemetry effectively. Several open-source options provide powerful visualization and analysis:

• Jaeger: Focused on distributed tracing, supported by the same organization (CNCF) as OpenTelemetry
• Prometheus: Industry-standard metrics storage and querying
• Grafana: Comprehensive visualization platform with both open-source and commercial offerings

These tools can be self-hosted, giving you complete control over your observability infrastructure.

The Community Behind OpenTelemetry

OpenTelemetry development happens through Special Interest Groups (SIGs) that meet regularly to review contributions, discuss problems, and plan features. The project maintains strict governance rules, including limiting representation from any single company on key committees to ensure true vendor neutrality.

The Ruby SIG welcomes new contributors, particularly those who:

• Use OpenTelemetry and encounter issues or limitations
• Want to add instrumentation for gems not currently supported
• Have experience to share about real-world usage

Contributing doesn’t require deep expertise—many contributors start by reporting bugs or asking questions, then gradually become more involved.

Looking Forward: Structured Logging and Profiles

The observability landscape continues evolving. Rails 8 introduces structured event logging with the Event Reporter, which provides built-in support for log attributes—something OpenTelemetry has championed. This alignment between Rails conventions and OpenTelemetry standards will make adoption even smoother.

Additionally, OpenTelemetry is working on profiles as a fourth signal type. Elastic donated their continuous profiling specification to the project, and future versions will include standardized profiling alongside traces, metrics, and logs.

Building Human-Friendly Systems

At its core, observability is about empathy—empathy for users experiencing issues, for teammates debugging unfamiliar code, and for future engineers maintaining the system. When you add observability thoughtfully, you’re not just collecting data; you’re creating a shared understanding of how your system works and how people interact with it.

This human-centered approach means considering questions like:

• What problems do my users face?
• What will I need to know when debugging at 2 AM?
• How can I help my team understand what this feature does?
• What would future maintainers want to know?

Getting Started Today

If you’re ready to explore OpenTelemetry in your Rails application, start small:

1. Review the Rails Guides documentation on Active Support Notifications to understand what’s already instrumented
2. Install the OpenTelemetry gems and configure basic tracing
3. Choose one area of your application—perhaps background jobs or a critical controller action
4. Add custom instrumentation and observe the results locally
5. Gradually expand coverage as you become comfortable with the tools

The OpenTelemetry Ruby community welcomes developers at all experience levels. Whether you’re troubleshooting production issues, optimizing performance, or just curious about what’s happening inside your application, these tools provide the visibility you need to build better software.

Resources and Next Steps

• OpenTelemetry Ruby GitHub: Find the community repository with links to all SIGs and meeting schedules
• Rails Guides: Review the comprehensive list of built-in Active Support Notifications
• OpenTelemetry Documentation: Explore semantic conventions and specifications
• Ruby SIG Meetings: Join the weekly or bi-weekly meetings to connect with other Ruby developers

The journey toward better observability starts with a single step. Whether that’s instrumenting your first background job, adding custom spans to a critical code path, or simply installing the gems and exploring what they reveal, you’re investing in a more maintainable, understandable system for everyone who works with it.