About Me

• Uchio Kondo / @udzura
• SmartHR, Inc.
  • Technology Platform Division
  • Favorite SmartHR feature: Personnel Orders
  • Favorite Rust type: Cell<T>
• Fukuoka.rb / Fukuoka.wasm

Today's Route

Before We Begin

• This talk is more a collection of questions than a single conclusion.
• Some details have been adapted; the technical substance is unchanged.
• Reasonable engineers may make different calls. Let's discuss them in the hallway or at the social event.

Rails and ActiveStorage

• One of the most basic requirements of a web service
• Today, Rails has ActiveStorage
  • Introduced in Rails 5.2 (2018)

B.A. (Before ActiveStorage)

• Before that, no "Rail" was laid for files
• Many gems competed
  • attachment_fu
  • Paperclip (deprecated in 2018)
  • CarrierWave / Dragonfly / Refile
  • Shrine

When It Was Built

• SmartHR's application got its first commit in February 2015
• A world before ActiveStorage

9 Years Later

• Migrated once from Paperclip, then about 9 years on CarrierWave
• 32 Uploader classes · 87 mount_uploader calls
• Up to 8 images per model
• CarrierWave is on an older version; upgrades require compatibility work
• Other dependencies, especially Ruby and Rails, are kept near the latest versions

Symptoms

Start with Smaller Problems

• “Replace CarrierWave” is too large to tackle at once
• Separate file access, history behavior, and other dependencies
• Choose a change small enough to implement and verify

Split the Problem into Three Qualities

Set Priorities Across Three Qualities

Two Main Problems Today

Given our current situation, we focus on:

1. File paths: keep saved files reachable
   • Fix fragile file paths and make them stable
2. History: avoid needless uploads when copying records
   • Tangled history logic slows down uploads

Third, briefly: maintainability — simplify dependencies.

Why Do File Paths Matter?

• Past incidents involved incorrect file references
• No files were lost in these incidents
• All these incidents were mitigated and resolved; affected customers were informed
• Reliability comes first for HR documents

Prerequisite: CarrierWave Lifecycle

On a validation retry, a Cache ID can restore the temporary file.

CarrierWave: Cache

CarrierWave: Store

File Paths Can Be Overridden Easily

• Convenient customization through store_dir
• Include model data such as an organization key or profile key
• Changes to that data or the method can make file paths fragile

# Profile image example; attribute names are illustrative
def store_dir
  "profile_images/#{model.organization_key}/#{model.profile_key}"
end

Complexity Leads to a Time-Based if

def store_dir
  if after_hotfix?   # updated after a certain date?
    new_store_dir
  else
    legacy_store_dir # kept for older files
  end
end

• Paths are not kept as snapshots, so we cannot change the logic

The Problem and the Two Fixes

• Changing model state or path rules can produce incorrect file references
• In these incidents, the files remained intact; the reference data was incorrect

Fix #1: Record cache_path

• Between cache and store, the calculated cache path could change
  • e.g., code where the path changes depending on whether an ID exists
• This caused multiple incidents in the past
• Fix: save the path at cache time, and never recalculate it

Cache: Before

Cache: After

Cache: Result

• Completed at the end of 2025, as our first improvement
• Incidents caused by cache_path dropped to zero
• Next: prevent store_path incidents from recurring

Fix #2: Record store_path

• We could change the identifier format, but chose a dedicated column
  • On read, prefer that column; otherwise, calculate as before
  • Backfill existing data by recalculation
• Start with the most problematic Uploader, then expand

Store: Record the Saved Path

class ApplicationUploader < CarrierWave::Uploader::Base
  after :store, :persist_object_store_path

  def persist_object_store_path(*)
    return if should_not_store_full_path?

    persisted_store_path = object_store_path_to_persist
    save_object_store_path(persisted_store_path)
    log_if_mismatch
  end
end

• Right after store, record the path in the dedicated column

NOTE: Simplified from the actual code.

Store: Read the Recorded Path

class ApplicationUploader < CarrierWave::Uploader::Base
  def store_path(*args)
    calculated_store_path = super(*args)
    persisted_store_path = resolve_persisted_store_path
    log_if_mismatch
    return calculated_store_path if persisted_store_path.nil?
    use_persisted_store_path ? persisted_store_path : calculated_store_path
  end
end

• No record? Use the calculated path, as before
• Log any mismatch between the calculated and recorded paths

NOTE: Simplified from the actual code.

Permanent Data Needs a Safe Rollout

• A wrong stored path can outlive the rollout
• Separate recording paths from using recorded paths
• Verify rollback as well as the forward migration

Flipper: Roll Out and Roll Back

• Use Flipper to enable write/read flags for selected tenants
• Add a dedicated column alongside the existing identifier
• Include rollback checks in QA

Monitor the Rollout with BigQuery

• Application data was periodically synced to BigQuery
• We queried that copy to monitor the file path rollout

-- Daily check on the BigQuery copy (simplified)
SELECT id, tenant_id, avatar, avatar_object_path
FROM app_mirror.employee_profiles
WHERE
  -- a file without a path, or a path without a file
  (COALESCE(avatar, '') = '') != (avatar_object_path IS NULL)
  -- or the path does not end with the file name
  OR NOT ENDS_WITH(avatar_object_path, CONCAT('/', avatar))

NOTE: Simplified from the actual code.

QA: Users Should See No Change

• All AI-assisted QA ran in staging with no customer data
• Features × preconditions create many test cases
• Define user stories and expected results with AI
• Keep automated tests and check human-visible evidence

QA: Stories Become Evidence

QA: Results and Human Review

• About 30 large stories, checked in 2 days by about 3 people
• Screenshots, logs, verdicts, and reasons are reviewed in PRs
• Existing tests are kept; automated coverage is strengthened
• The value: more cases checked, with a human final review

Store: Current Status

• Some images have fully switched to fixed paths
• We will expand to all images, step by step, the same way

File Paths: What Matters

• Record a stable reference to each saved file
• Separate writes and reads to make the rollout controllable
• Test the way back, not only the way forward

Making Images and History Work

• Employee history is essential to our service
• Handling images is essential too
• Combining them requires extra care

Bitemporal Data Model and Us

History Copying Has Side Effects

Our history-copy workflow + CarrierWave's assignment behavior
can upload the same file again.

The Fix: Mark History Copies

In the PR: CurrentAttributes carries
the “copy in progress” state into hooks.

CurrentAttributes: Into the Hook

# Per-request state, reset by Rails
class HistoryCopy < AS::CurrentAttributes
  attribute :in_progress, default: false
end

# In ApplicationUploader
def self.copying_history
  HistoryCopy.in_progress = true
  yield
ensure # reset, even on errors
  HistoryCopy.in_progress = false
end

# In ApplicationUploader
# Assignment runs cache!; skip it during copy
def cache!(*)
  return if HistoryCopy.in_progress

  super
end

# Usage
ApplicationUploader.copying_history do
  record.copy_history!
end

NOTE: Simplified from the actual code.

History: Measuring the Upcoming PR

Local run of a near-release PR: 17.0s → 9.6s (−44%) · GCS calls 39 → 22

History: Where the Time Went

Two spans disappear: rebuilding the history row and the repeated store

History: What Matters

• Make the difference between copying a reference and uploading a file explicit
• Keep lifecycle side effects under application control
• Validate the history behavior before rolling out
• The PR is nearly ready; the production rollout is next

Preprocessing: More Room to Improve

Separate local PoC: 17.9s → 5.8s · fewer resized images, fewer GCS uploads

Two Blockers to Upgrading

1. Tight coupling
   • Application code depends on CarrierWave internals
2. Incompatibilities
   • A version upgrade changes behavior we rely on

Current: Coupled Image Responsibilities

Image Operations: Find the Boundary

Nearly 300 call sites depend on CarrierWave.

user.avatar.present?

CarrierWaveCompatLayer.attached?(
  user, :avatar
)

Same behavior, an explicit boundary.
Migrate callers incrementally; extraction is under analysis.

Image Operations: Wrap the API

Incompatibilities: Investigated

• We have investigated the incompatibilities in the CarrierWave upgrade
• The next challenge is how to roll it out safely
• Understanding the differences is the starting point

The Same Rollback Strategy Won't Fit

• For file paths, we switched writes and reads separately
• A library upgrade changes the underlying implementation
• We cannot reuse the same flag-based rollback strategy

Reuse What We Learned

• AI-assisted QA may help check compatibility
• Gradual rollout may help limit the impact
• Build on our file-path migration experience

To be updated.

Address Both Upgrade Blockers

1. Tight coupling → clearer boundaries
   • Wrap library-specific behavior; migrate callers gradually
2. Incompatibilities → a safe upgrade plan
   • Investigation done; adapt our QA and rollout experience

What Could Replace CarrierWave?

No Decision Yet

• Upgrading CarrierWave and replacing it are both options
• Catching up to a newer version may be the practical next step
• First, simplify the behavior we must preserve
• Keep application needs separate from library internals