Oscaro.com is the leading online retailer of new and used automotive parts. The company has experienced double-digit growth since its creation in 2001, and achieved 300 million euros in revenue last year. It has a presence in France, Spain, Belgium and the United States and their website offers over 450 000 parts for more than 70,000 different models to their customers.
To handle this demand, a big data approach was necessary to best meet the expectations of Oscaro’s customers: since a web tracker was put on their e-commerce website, linking navigation data to the products catalog, Oscaro generates up to 5TB of data per year!
On the back-end, the volume of data was growing 150% to 200% per year, which was no longer compatible with standard database architectures (RDBMS / OLAP cube).
For the front-end, it had become increasingly difficult to access data from multiple sources, and running comparative time analysis was very time-consuming and complex to carry out on such large volumes. The Information System was becoming increasingly asymmetrical; on one side, technical users managed to cross analyze web traffic data with the original OLAP cube sales data, but on the other side, less technical business users who didn’t know how to code, couldn’t get answers to their questions.
To manage the data growth and analytics, Romain Nio, Data Management Team Leader, and François Dumarest, Pricing Project Manager for Oscaro, opted for a “Big data” oriented and "scalable" architecture with modern cloud technologies.
They were drawn to Google BigQuery, allowing business teams to query very large volume of data within seconds, without having to worry about the underlying technical architecture. BigQuery also significantly reduces the development cycle through a very simple SQL-like syntax, and makes it quick and easy to input data via flat files. The data input is done in batches for better aggregation from heterogeneous data coming from 80 different sources, mainly using Hadoop Google Cloud. The data is cleaned and joins are made in Hadoop to link web traffic data and orders. All this is then pushed into Google BigQuery.
Oscaro was able to significantly improve performance through a quick and easy deployment of massive additional computing power, while optimizing infrastructure costs with their choice of the new architecture.
On the front-end
BigQuery is not intended to be a standalone platform for the business users, François and Romain decided to use BIME, a cloud-based Business Intelligence solution, that can query BigQuery live in the cloud.
The BigQuery & BIME combination means high performance and a very intuitive interface for the analysis and dashboard publishing, even with large volumes of heterogeneous data and real-time online access from the browser.
"We first focused on the implementation of Google BigQuery and then, deploying BIME naturally made sense as a powerful analytical solution: the 100% cloud technology combination was unbeatable because both were quick to set up and easier to handle than the OLAP cube our end-users had previously used since the beginning of Oscaro’s journey. " said Romain Nio.
The power of the 100% cloud combo BIME + BigQuery : What changed
"Our analyses today are deeper than ever: our business users from marketing and sales, can analyze our performance more quickly whether it’s related to web traffic, customer acquisition costs per channel (eg. Adwords), product margins, logistics data (response time), and also call center data for customer satisfaction... . In short, all the information that ensures the proper execution of our value chain," adds Francois.
The simplicty with which they got BIME up and running allowed several analysts, including François Dumarest, to quickly implement their dashboards. They connect from a single interface to multiple data sources, including BigQuery and share their new analysis with internal teams.
The analysis carried out within the OLAP cube still remains, but new sources of information are now accessible through BIME, facilitating its adoption and faster transition for the business users.
The implementation of BigQuery + BIME took less than 6 months. During the first 3 months, transfer tests from Hadoop to BigQuery were made, and the analytical capabilities of BIME were used to run data quality checks. Then came the business analysis phase. Building and optimizing standard and custom dashboards shared with users was done in just two months.
"Even before BIME was used as an operational BI solution for our departments, it was critical during our BigQuery implementation. BIME immediately allowed us to see data inconsistencies and time gaps - basically the health of our data. With a few clicks and a few cross-checks in BIME, we could see if the numbers fed into BigQuery were accurate and matching what was in our legacy system.
We needed BIME for the core business analysis, but we also used it upstream. BIME clearly facilitated Google BigQuery's integration. Anytime we needed help, the BIME Support Team was conviently available online” says Romain.
François and Romain had also previously tested BigQuery with other vendors such as Tableau Software and QlikView, but they were not as responsive as they are not 100% cloud-based (time for server installation & configuration: a whole R&D project in itself). Also, being very affordable, the BIME + BigQuery combo was a perfect fit for Oscaro.
"Compared to other solutions, with BIME, we did not have to install an additional BI infrastructure. We did not have to pull resources from our IT Team. As BIME evolves over time, we have access to new features & functionalities every week, which is not possible with on-prem BI", says Francois.
Exchanges between different departments are also more relevant because the data isn’t asymetrical anymore and can’t be interpreted differently (there is now a single definition for an order error, abandoned transactions,... etc.) - everyone speaks the same language via a more qualitative solution, much easier to present than an Excel file.
The use of these technologies, hiding the complexity, allowed us to spend more time upstream on managing the data cleansing and on the documentation detailing the definition and characteristics of the data and what it represents, reducing significantly the number of errors compared to before.
"The speed of anaylsis has completely changed. Before using BIME, we had to reprocess data from Hadoop to allow the business user to actually query it, which used to take us at least half a day of work - whereas with BIME, it only takes 5 minutes” says Francois.
The next step is to deal with purchase and financial data. "When we put this in BIME, with all the cross-analysis imaginable, it will get even more interesting. Today, it takes time to transfer a new type of data project, from the old system to BigQuery. Once this is done, however, the easy “plug'n'play” analysis in BIME will be possible for users and it will be available almost immediately" adds Romain.
The current challenge is linked to the business users’ habits, a classic situation when modernizing the framework of an Information System : "Users love their Excel extractions and their “Vlookups”. But we show them that BIME is a new way of looking at their data, with a wider scope. Deeper analysis is possible simply using BIME’s calculation engine, and BIME has automated layouts, making dashboards easily sharable, updated in real time, even on mobile when on the go."
Among their favorite features, François and Romain highlighted the following:
The filter capabilities
for M/M-1 temporal analysis with just a few clicks
for a simple way to share dashboards to different distribution groups without the dashboard designer having to duplicate or recreate anything
for very personalized interactive questions from readers through the list of criteria available to them (timeframes that suit them, changing measures on the fly that immediately update all dashboards, ...)
The advanced visualization types such as “Sunburst”
The simplicity of handling of dashboards for readers
The simplicity of the BigQuery connection with BIME... at record speed.
"The dashboard designer is saving time with BIME and our teams on the field are also saving time with their analysis."
"The fact that BIME is a known pure player with 100% cloud DNA and their responsive platform allow us to consider mobile BI projects without having to embark on a new battle or another IT project," concludes Romain.
A big thank you from the BIME Team to Romain Nio, Francois Dumarest and the Oscaro Team.