The Importance of Testing: How one test applied to two email sends resulted in different audience responses

At MarketingExperiments, sister company of MarketingSherpa and parent company of MECLABS, we believe in testing. Our Web clinics are a testament to this belief – every month we share research that is designed to help marketers do their jobs better.

This culture of testing encouraged me to run my own test.

First, I ran the test on a MarketingSherpa send. After the results of that test came back, the same test was then applied to a MarketingExperiments’ newsletter.

By running virtually the same test twice for two different audiences not only did I learn more about the preferences of ourreaders, but I also learned how incredibly important it is to test, even when you are sure you know what the results will be.


The MarketingSherpa test

As the copy editor at MECLABS, I get to see all of the copy produced by both MarketingExperiments and MarketingSherpa. One of my responsibilities is overseeing the email newsletter sends. Every Monday, MarketingSherpa sends out a Best of the Week newsletter which features the most popular articles of the previous week and introduces the new book for the MarketingSherpa Book Giveaway.

The copy in these newsletters was extensive. Every article listed had a full summary which, as a reader, I imagined would seem overwhelming when opening this email on a Monday morning.

With the help of Selena Blue, Manager of Editoral Content, and Daniel Burstein, Director of Editorial Content, I decided to change reduce the length of the Best of the Week newsletter and use an A/B split test to determine which format our readers preferred.

Below is a side-by-side of the Control and the Treatment. 


After running this test for six weeks, there was a definite trend of higher total and unique clickthrough happening for the shorter treatment versions. My hypothesis proved to be true — less copy in the email meant higher engagement rates with readers.


The MarketingExperiments’ test

Based on the success of the MarketingSherpa test, I thought it would be interesting to run the same test on MarketingExperiments’ Best of the Month newsletter send, which is sent to readers on the first of every month. Similar to the Best of the Week email send, the Best of the Month features the most popular articles of the month, gives a recap of the latest Web clinic and provides a description of the upcoming Web clinic. This was done with the help of Ken Bowen, Managing Editor, MarketingExperiments.

The Best of the Month send has more information in it, so it is naturally a longer newsletter than the Best of the Week email previously tested. Therefore, it felt like a natural mental leap to assume that the shorter Treatment would once again get higher clickthrough rates as it seemed to be more reader-friendly.

As you can see with the below Control of the Best of the Month from August, there was a lot of copy in these sends.


For the Treatment, we once again cut the article summaries. In addition to losing this copy, we also cut the recap of the previous month’s research and instead just had the upcoming month’s Web clinic description. We ran this A/B split test for three months.


Due to the results of the Best of the Week testing, I knew what the results of this testing would be. If one audience responded positively the lessened copy, certainly so would another audience. The fact that we were virtually re-running the same test seemed to me to just be a formality.

Spoiler Alert: This is why testing is so imperative.

If I had applied the results from one test to all newsletter sends, I would have been doing our readers a disservice. Below are the results broken down by month.


Of the three months of testing, only one month validated. Although the Treatment had slightly higher clickthrough rates, it wasn’t significant enough to regularly validate as it did with the Best of the Week send. It appears that readers of the Best of the Month newsletter may find value in the summaries and extended copy that the Control version had.

Testing allowed us to change the MarketingSherpa Best of the Week send to better serve our readers and also learn that the same changes made to the Best of the Month send did not make a statistically significant difference to MarketingExperiments’ readers.



Testing is so important. I work for a company that values testing and is continually striving to best suit the needs of our readers. The editors and directors overseeing both MarketingSherpa and MarketingExperiments were very open to my experimenting with the format of a send that has been with the company much longer than I have.

And while the change of the format itself is important, so was the testing involved. It allowed me to really see how our readers were digesting the information that I was sending to them on a weekly or monthly basis. To better understand the patterns and behaviors of our audience only benefits me as a copy editor.

This meant that one audience preferred the lessened copy while the other may find value with more text . Without re-running the test, I would have blindly assumed that the MarketingExperiments audience had the same reading preferences as our MarketingSherpa audience.

Testing allows us to validate even that which we are sure to be true, and to humble us when this in fact turns out to be false.


You might also like

MarketingSherpa Summit 2016 — At the Bellagio in Las Vegas, February 22-24

A/B Testing: Ecommerce site’s 3,000 positive comments show why you can’t trust just one test

Testing and Optimization: Welcome send test results in 46% open rate for CNET


7 Types of Twitter Contests to Boost Your Reach And Engagement

Looking for more reach and engagement on Twitter? Try running a contest!

Contests can benefit your brand in a number of ways. They can increase your number of followers, improve your reach and visibility, encourage your followers to engage more with your content, generate positive sentiment and more.

If you’ve considered running a Twitter contest, but aren’t quite sure where to start, we’ve got 7 ideas for different types of contests you can run.

1. Photo contests

Photo contests are a great way to get your current follower base more engaged with your brand. You can ask them to take a photo of themselves with your product, snap a funny pic or share an image of their daily lives.

With a photo contest, be sure you either retweet or thank each entry – and when you select a winner, give it lots of fanfare on Twitter!

2. Creative/funny answer contests

Ask your followers to submit their most creative or funny answer to a question or scenario you tweet. It’s a good idea to include a hashtag as part of this campaign, so you can collect all of the responses in one place.

Again, retweet the best entries to show appreciation, and to encourage other followers to enter.

3. Retweet to enter contests

One of the simplest types of Twitter contests is the retweet to enter. All your followers have to do is retweet your “seed” contest tweet, and they will be entered to win!

As a brand, you only need to look at who retweeted that single tweet, and randomly select a winner. What could be easier than that?

4. Follow to win contests

Another dead-simple Twitter contest is the follow to win. You can announce this contest across other social media platforms, letting your followers elsewhere know that if they follow you on Twitter they could win a prize.

5. First-to-answer contests

For an ultra-short-term contest, why not give your fastest followers a reward? Ask a question, and give the prize to the first person to tweet the correct answer. This works well in short spurts, and if you run this type of contest regularly, your followers will be more likely to keep tuning in to your tweets so they don’t miss out.

6. Tweet a hashtag contests

Creating a branded hashtag for a specific contest is a good way to improve your reach while rewarding your followers. All they have to do is include the hashtag you create in a tweet that is relevant to the contest, and they will be entered to win.

7. Caption contests

If you’ve got a creative bunch of followers, you can ask them to write a funny, interesting or smart caption for a photo you tweet. You’ll get to select the best one of the bunch and reward them with a prize. As with the other creative contests, be sure to tweet the winning entry.

As you’re creating your contests, don’t forget to follow Twitter’s guidelines to keep them above-board, and be sure to track the results. But most of all, have fun!

Why Everyone Should Be Moving To HTTP/2

A Google spokesperson recently revealed that GoogleBot will soon support HTTP/2. Columnist Patrick Stox explains what this is and what it means for SEOs. The post Why Everyone Should Be Moving To HTTP/2 appeared first on Search Engine Land.


Please visit Search Engine Land for the full article.

Google has received nearly 350,000 URL removal requests so far



In May 2014, Google complied with the EU’s ruling that enabled citizens to request certain URLs to be removed from search results

A year and a half later, in its bi-annual Transparency Report, Google has announced it received 348,085 such requests, the bulk of them coming from France and Germany, followed by the UK, Spain and Italy

The total number of URLs Google has evaluated for removal is 1,234,092. Google does not give exact numbers on how many of those it actually removed, but it says that 42% of the URLs that have been “fully processed” have been removed Read more…

More about Google, Transparency Report, and Social Media

Instagram is testing support for multiple accounts on Android



Instagram may finally be adding support for multiple accounts.

The photo sharing app is testing support for multiple accounts for Android users. The feature appears to be limited to some Android users who are part of Instagram’s beta testing program, at least for now.

First spotted by Twitter users @fro_rogue — who reports the test has been live for some time — the feature allows users to add multiple accounts from the app’s main settings menu. Once added, you can browse and post from different feeds within the app Read more…

More about Facebook, Social Media, Tech, Android Apps, and Instagram

Madden NFL Mobile Receives Road to the Playoffs Update

Electronic Arts has released a new update for Madden NFL Mobile, as well as an infographic detailing stats about the game.

The update features new Road to the Playoffs content, which lets players follow along with the current NFL season in an attempt to make the playoffs. This feature gives players the ability to unlock Tom Brady, who has a 96 overall rating. Brady can be earned by completing the Road Master set.

This update also gives players the ability to play as blocking running backs. In addition, players can complete new League Quests by teaming up with league-mates to complete challenges, and Prestige Level players (those level 50 and up) now have access to alternate uniform selections.

EA will continue to add new Road to the Playoffs content to the game in November and December, with Super Bowl content coming at the end of the football season.

Since Madden NFL Mobile launched in 2014, 133 million season games, 46 million player-vs-player games and 23 million league games have been played. In addition, 615 million live events have been played. The game’s top offensive play is the HB Toss, which has been called 732 million times. For defense, the top play is ‘2 Man Under,’ which has been called 438 million times. Check out the infographic below for more.

Madden NFL Mobile is available to download for free on the iTunes App StoreGoogle Play and the Amazon Appstore.

Madden NFL Mobile Infographic

How Did NBA Stars Steph Curry, Kobe Bryant and Anthony Davis Start on Twitter?

NBA superstars such as Stephen Curry of the Golden State Warriors and Anthony Davis of the New Orleans Pelicans are fun to follow on Twitter for more than just basketball highlights.

Athletes are among the most-followed celebrities on social media, but everyone has to start somewhere. Just like the rest of us, these players had to pick a user name, register for an account and compose that first tweet.

In a Twitter Moment, Twitter recently surfaced the first tweets of NBA stars such as Dwyane Wade, John Wall and James Harden, as well as legendary center and funnyman Shaquille O’Neal.

Here’s a look at the Twitter debuts of some of your favorite NBA players. While some took to the platform with aplomb, others weren’t sure what to make of it at first.



Readers: Who is your favorite player to follow on Twitter?

Image courtesy of the Golden State Warriors on Facebook.


QuizUp Reveals Most Popular Trivia Topic in Each State

Plain Vanilla Games, creator of the QuizUp mobile trivia game, has analyzed the habits of American QuizUp players to discover which topics are the most popular in each state in the U.S.

The company analyzed million of games played by American users, and has determined the most popular topic in each state, and in Washington D.C., as well as the most popular topics in the entire country overall.

A blog post explains:

We set out to gather and present the data on the most popular trivia topics nationwide and in each state, analyzing millions of games played on American soil. We then ranked the 50 most popular topics in each state in order of total games played.

In the map below, we’ve presented the single topic that ranked higher in a given state than in any other. For instance, if a particular topic was ranked No. 5 in one state and the highest ranking it received in any other state was No. 11, we designated the topic as that state’s favorite on the basis of relative interest.

The state map reveals the popularity of major television shows and movies, as topics like Frozen and The Big Bang Theory (as examples) are the most popular topics in New Hampshire and Iowa, respectively. While some popular topics may seem unusual, others make complete sense. For instance, the most popular topic in Washington D.C. is U.S. Government, while the most popular topic in Washington state is the Seattle Seahawks.

In addition to breaking down topics by state, Plain Vanilla Games determined the top ten QuizUp categories across the country, with Logos and General Knowledge coming in at first and second place, followed by Disney Movies, Name the Food and Frozen in the top five.

Check out the full results in the infographics below.

QuizUp States

QuizUp America

5 Digital Trends Influencing the Future of Retail (Infographic)

Retail marketing has been heavily impacted by the presence of social media, and other advances in the digital space. An infographic from MediaMath provides five trends every marketer should be aware of going into the holiday season.

  1. Digital to influence the majority of in store sales. This year 64 percent of retail sales will be influenced by the digital space by the end of 2015, and 78 percent of digital shoppers research purchases online before making in store purchases. Integrating online and offline campaigns, and using multi-touch attribution can help ease the transition for customers.
  2. Mobile is now the majority of digital ad spending. 51.5 percent of all digital ad spending in the retail space this year will be targeted for mobile, and the spend is predicted to be $6.65 billion. With 67 percent of shoppers shopping across multiple devices, it’s imperative to create a good user experience across all platforms.
  3. Direct publisher deals going programmatic. Spending on private marketplace (PMP) ad solutions grew 381 percent in 2015. Despite this growth, only 14 percent of retail digital display ads are served through such a solution. PMP marketing provides opportunities to market in a brand safe, guaranteed, and viewable way throughout your campaigns.
  4. Material part of digital ad spend in 2015. Video is rapidly becoming the most engaging marketing tool online. 29 percent of the digital display spend in retail will be for video advertising, and ad execs believe that the ability to target video content is the “most valuable digital advertising feature for their clients.” When video is so ubiquitous, and targeting so powerful, it would be wise to invest in the space.
  5. Closed-loop attribution is the next logical step. 15 to 20 percent of retailers are using multi-touch attribution vendors, and a further 20 to 25 percent are in active discussions with such providers. It seems that marketing convergence could be the way forward for marketers as it provides greater control over the message, and the atmosphere where the ads are served.

For more information view the infographic below.