A company store assortment experiment should answer a narrow buying decision: whether a new product deserves more inventory, a different position, a revised offer, or a clean stop. It should not turn a few orders into a claim about customer preference. The practical work is to make exposure, availability, price, cost, and eligibility comparable enough that a merchandising owner can explain the next action.
Download the English XLSX scorecard. It contains Inputs, Exposure, Sales, and Decision sheets with illustrative rows. Replace the amber cells with approved evidence. The workbook reports descriptive results and an evidence gate; it does not calculate statistical significance.
Why assortment experiments fail before anyone calculates conversion
Inputs to reconcile before comparing the mug and bottle
| Input | Evidence to record | When to hold the decision |
|---|---|---|
| Eligible audience | Approved recipients per variant, same period | Audience rules differ |
| Availability | In-stock hours divided by planned hours | One variant sold out during peak traffic |
| Net contribution | Revenue after refunds less attributable costs | Refund window remains open |
The most common failure is not a bad formula. It is an unfair opportunity to sell. A new bottle may appear lower in a category, spend nine hours out of stock, have fewer eligible viewers, or carry a different delivery promise than the incumbent mug. If the team divides orders by all site sessions, the result looks precise while hiding those differences.
Assortment work also mixes several decisions that should remain separate. A launch decision asks whether to keep learning. A replenishment decision asks how much to buy. A retirement decision asks whether accumulated evidence supports removing a product. A catalog architecture decision asks how many choices the store should show. One short pilot cannot answer all four.
The PPAI and Oxford Economics market study published on September 22, 2026 provides useful context about the breadth of branded merchandise spending and purchasing channels. Its figures describe a market and economic footprint, not the conversion rate of any one company store or the likely result of a bottle test. Market context may justify disciplined learning; it cannot supply a missing denominator.
Start by writing one reversible decision. For example: "After a fourteen-day pilot, the merchandising lead will either expand the bottle, revise its offer, extend the test, or stop it." That sentence identifies the time window, owner, and available actions. It also leaves room for an honest inconclusive outcome.
Write a hypothesis that can survive an inconclusive result
A useful hypothesis names the audience, product difference, comparison, and measurement period. "The sustainable bottle will be popular" is not enough. A better statement is: "Among eligible US employees who see a comparable category position during the same fourteen-day window, the bottle will generate enough valid exposure, contribution, and operational reliability to justify one replenishment cycle alongside the mug."
This wording avoids three traps. First, it does not promise that the bottle will win. Second, it does not treat a sustainability label as evidence of superior environmental performance. The Federal Trade Commission's Green Guides summary says broad, unqualified environmental benefit claims are difficult to substantiate and recommends clear, specific qualifications. Third, it connects the test to an operational action instead of a vanity metric.
Record the alternative before launch. The bottle might replace the mug, sit beside it, move to a different market, receive a price or content revision, or stop. Each path has different evidence needs. Replacement requires confidence that the comparison was fair and that displaced demand will not create support or size problems. Parallel listing may tolerate more uncertainty but consumes catalog space and working capital. A local-market revision may be appropriate when labeling, size, material, or delivery expectations differ.
The scorecard keeps the hypothesis beside the editable commercial inputs. That placement matters. If price, unit cost, inventory, or the eligible audience changes after launch, the owner can see that the original question changed too. Do not silently move the goalposts and call the revised test a continuation.
Define eligible audience before counting product views
Eligibility is the population that could actually choose the item under the program rules. It is not every employee, every site visitor, or every person who received an email. Country availability, budget, role, store access, redemption balance, and product restrictions may all narrow the population.
Use a dated eligibility snapshot. The program owner should preserve the rule version, the count of eligible people, and the exclusions applied. If a person could not see the bottle because it was unavailable in their country, that person should not inflate the bottle's denominator. If the incumbent mug appeared to everyone but the bottle appeared only to a pilot group, report each denominator separately.
The Giftpack Help Center describes configurable company-defined budgets, eligibility, catalog rules, country availability, and recipient choice. Those capabilities can support a bounded assortment scenario when the actual program is configured that way. They do not prove that a specific catalog, integration, audience, or product is currently enabled. The owner must verify the live configuration and preserve a dated export or approved rule record.
Minimize personal data. An assortment test usually needs a program key, market, eligibility state, exposure event, order event, and exception status. It usually does not need sensitive traits or speculative profiles. If demographic fairness is part of a separately approved analysis, legal and privacy owners should define the permitted fields, minimum group sizes, retention period, and access controls before data is collected.
Acceptance evidence for this step is simple: one approved eligibility definition, one effective date, one source record, and one accountable owner. Without those items, a later conversion rate has no stable population behind it.
Build a valid exposure denominator
Exposure means a qualified opportunity to consider a product. A raw product-page view may not be qualified. A page could load while the item is out of stock, show a broken image, display a misleading price, appear below an accordion, or arrive from an internal test account. The scorecard therefore separates raw views, out-of-stock views, and other invalid views.
The workbook uses:
Valid exposure = MAX(0, raw product views - out-of-stock views - other documented invalid views)
It then calculates:
Exposure rate = valid exposure / eligible audience
These formulas are deliberately transparent. They do not estimate unseen impressions or manufacture a comparable denominator. If the event design cannot distinguish an out-of-stock page view from a sellable view, mark the exposure limitation and redesign the event before making a winner claim.
Placement should be documented as part of exposure. Record category position, default sort, search availability, navigation path, image state, description state, price, shipping promise, and whether the item was included in the same campaign. A position change during the test is an intervention. Either freeze it or treat the periods separately.
The GOV.UK guidance on A/B testing and comparative studies frames a comparison as a test of design choices and recommends reviewing the hypothesis or variations when results are inconclusive. A company store does not become a controlled clinical trial merely by using an A/B label. The useful lesson is narrower: compare like with like where possible, define the outcome in advance, and accept that an inconclusive result calls for another decision rather than a forced winner.
Keep availability separate from interest
An unavailable item cannot earn a fair order. The scorecard records outage hours and total window hours, then calculates availability as one minus outage hours divided by total window hours. The illustrative bottle is unavailable for nine of 168 hours, producing 94.6% availability. The example evidence gate requires at least 95%, so the workbook returns "insufficient evidence" even though the bottle's contribution is higher.
That boundary is an editable operating assumption, not an industry benchmark. The team should set it before launch based on the decision risk. A low-cost reversible assortment test may tolerate a small outage. A major procurement commitment may need near-continuous availability and stronger controls.
Track more than a binary stock state when the store supports variants. A bottle can be technically available while the preferred color is gone. Apparel can be available while common sizes are missing. Country routing can show the product in one market and suppress it in another. Define the availability unit that matches the buying decision: product, variant, size, country, or fulfillment node.
When an outage occurs, preserve the raw event rows and tag the affected interval. Do not delete the period to improve the chart. The decision owner should choose among extending the test, narrowing the inference, repeating the test, or ending it as inconclusive. The exception and disposition belong in the acceptance record.
Read sales, sell-through, and contribution together
Orders answer only one part of the decision. A product can sell more units because it is cheaper, occupy more exposure because it is promoted, or show a high sell-through rate because the starting inventory was too small. The scorecard presents several descriptive measures side by side.
| Measure | Formula | Decision use | Main limitation |
|---|---|---|---|
| Observed conversion | Units sold divided by valid exposure | Describes response among recorded opportunities | Depends on valid, comparable exposure |
| Sell-through | Units sold divided by starting inventory | Shows how much stocked quantity moved | A small initial buy can inflate the rate |
| Net sales | Gross sales minus refunds | Removes recorded refunds from revenue | Later returns may be outside the window |
| Contribution | Net sales minus product cost minus other variable cost | Shows a traceable commercial contribution | Fixed costs and capital costs may be omitted |
| Availability | One minus outage hours divided by test hours | Tests whether stock constrained demand | Product-level availability may hide variant gaps |
Use consistent cost definitions. Product cost, decoration, pick-and-pack, payment fees, and incremental shipping may belong in variable cost, depending on the decision. Document the included fields once. Do not compare a fully loaded bottle cost with a partial mug cost.
Contribution is not return on investment. The workbook does not include fixed platform fees, staff time, procurement effort, sample cost, storage risk, or the opportunity cost of displaced inventory unless the user adds them. If those costs could change the decision, extend the calculation and label the methodology.
Use an evidence gate before discussing a winner
The evidence gate asks whether the comparison is usable before anyone ranks products. The example checks valid exposure and availability, while the article asks the owner to review placement, eligibility, price, and data completeness as well. Passing the gate does not prove causation. It means the team has enough documented comparability to discuss the descriptive result.
A practical gate can include:
- Minimum valid exposure for each variant, set before launch.
- Minimum availability at the decision unit, such as product or size.
- Same eligibility rule and measurement window.
- Documented placement and campaign treatment.
- Complete price, refund, and cost fields.
- No unresolved duplicate orders, internal tests, or event replay.
- A recorded exception disposition for every material break.
If any required condition fails, the result should be "insufficient evidence," not "B lost." That status protects future learning. It tells the next owner what to fix: inventory, instrumentation, placement, price design, or sample coverage.
Do not set the gate after seeing the result. A post hoc threshold can always be tuned to admit a preferred answer. Record the assumptions, approvers, and effective time in the test brief. If the business deliberately changes a threshold, create a new version and explain why.
Assign owners and preserve a decision trail
The merchandising owner defines the decision and product alternatives. Finance approves the cost definition and checks refunds, fees, and contribution. Store operations verifies inventory, placement, and fulfillment exceptions. Data or analytics owns event definitions and reconciliation. Privacy, legal, or security participates when the test uses regulated fields, environmental claims, or a new data flow. Recipient support records confusion, substitutions, and failed deliveries that may explain the numbers.
Use a short operating sequence:
- Approve the hypothesis, alternatives, audience, window, and evidence gate.
- Freeze or version the price, content, placement, and eligibility rules.
- Test exposure and order events with nonproduction records that are excluded from analysis.
- Record starting inventory and every outage or variant gap.
- Reconcile valid exposure to orders, refunds, and fulfillment exceptions.
- Review the evidence gate before comparing outcomes.
- Document the decision, limitations, owner, and next review date.
Acceptance should include the signed decision record, dated source extracts, formula version, exception log, and exact workbook version. A screenshot is not enough because it cannot show the inputs and formulas that produced the status.
Worked case A: a bottle appears stronger but fails the availability gate
Consider an illustrative test, not a Giftpack customer result. A company store compares an incumbent mug with a new bottle for one week. Both have 1,200 eligible people. The mug records 850 raw views, ten out-of-stock views, and fifteen other invalid views, leaving 825 valid exposures. The bottle records 830 raw views, fifty out-of-stock views, and twenty other invalid views, leaving 760 valid exposures.
The mug sells 82 units and the bottle sells 71. Observed conversion is about 9.9% for the mug and 9.3% for the bottle. The bottle still shows higher illustrative contribution because its price and cost structure differ. A simple revenue ranking might favor the bottle; a simple conversion ranking might favor the mug.
The evidence gate prevents either conclusion. The bottle is available for only 94.6% of the week, below the prewritten 95% boundary. Its fifty out-of-stock views also show that the unavailable interval touched the denominator. The correct action is to repair availability and extend or repeat the test, not to declare a winner.
The recovery record should identify the affected hours, SKUs or variants, cause, inventory correction, and new observation window. The original rows remain preserved. If the team cannot restore comparable availability, it may narrow the decision to operational readiness rather than preference.
Worked case B: a local-market split is too small to support a winner claim
Consider a second illustrative case. A global store offers a notebook in Japan and a compact pouch in Taiwan. Traffic is small, and each item has different local labeling and delivery constraints. The first week produces only a handful of valid exposures per item.
The team should not pool the markets merely to increase the number. Japan and Taiwan have different audiences, product expectations, labeling duties, delivery conditions, and possibly prices. Combining them can erase the reason the variants were localized.
Instead, mark the quantitative result unknown. Review whether the instrumentation worked, whether the offer was readable, and whether the local product could remain available. The owner can extend the window, recruit an approved broader audience, use a carefully documented qualitative review, or stop the test. Each choice has a cost. Extension delays the decision and holds inventory. A broader audience may change the target population. Qualitative evidence can explain usability but cannot substitute for sales behavior.
Acceptance evidence is the reasoned next action, not a fabricated percentage. The scorecard supports this by making the evidence gate visible and by leaving descriptive measures separate from the decision.
Respect local product and environmental claim boundaries
An assortment experiment does not suspend product rules. A sustainability-themed bottle still needs substantiated, specific claims. A textile, container, electrical item, or children's product may have market-specific labeling or safety duties. The store owner and supplier should identify the responsible party and approved product evidence before the item enters the test.
For a US-facing environmental claim, use the FTC guidance as a starting point and obtain appropriate legal review. "Made with 30% recycled material" is materially different from an unqualified "green" claim. The scorecard can record which claim version appeared during the test, but it cannot validate the underlying material evidence.
For Taiwan, Japan, and Korea, the localized editions point to official authorities and explain only the relevant decision boundary. The merchandising team should not copy a US statement into another market. Product scope, responsible entity, language, placement, and required evidence need local verification.
If a claim or label changes during the window, version the content and assess whether the change altered demand. A material description change may require a new test rather than a blended result.
Where Giftpack can fit without taking over the decision
Giftpack can be considered when a company needs an execution layer for an approved audience, budget, country set, and catalog rule. The current official FAQ describes recipient choice, configurable eligibility, country availability, and catalog controls. A hypothetical company could use those controls to present a bounded assortment, then reconcile approved events, orders, and fulfillment exceptions in its own decision record.
That scenario must be verified against the actual program. It does not establish a native analytics integration, automatic experiment assignment, environmental certification, or guaranteed improvement. The buyer still owns the hypothesis, lawful data use, cost definition, product evidence, and decision.
Teams designing the data trail may also use the corporate gifting data governance guide to frame consent, retention, and regional access questions. Treat that article as operational context, not legal advice.
Make the next action more credible than the winner label
A useful assortment experiment ends with a traceable action: expand, revise, extend, localize, or stop. The company store assortment experiment scorecard keeps the decision connected to eligible audience, valid exposure, availability, sales, refunds, cost, and exceptions. Its worked rows are hypothetical and its thresholds are editable assumptions, not benchmarks.
The strongest outcome is sometimes "insufficient evidence." That result prevents a stock problem, placement difference, or tiny local sample from becoming a false preference claim. Preserve the raw rows, record the limitation, and decide what evidence is worth buying next.
When Giftpack is a verified fit for the approved program, its enterprise gifting capabilities can support catalog and fulfillment execution. The merchandising, finance, privacy, legal, and operations owners remain responsible for the experiment design and the final assortment decision.