How An Ad /Content Targeting Engine Is Built

We often come across ads/stories when we access Facebook, Instagram or other web apps/games like Candy Crush etc.

Ever wondered why and how this relevant content reaches us?

If we try to devise a user story as a Product Manager, we’ll surely be able to come up with something in terms of who to target, how to target, when to target and where to target, with what kinds of ad.

So, skipping this step for now, we’ll try to look at what an Advertising Engine’s high-level architecture would look like.

In order to arrive to our vision, we need to start with a set of mandatory activities for laying out the foundation of our product. The foundation shall include the following list of capabilities:

1. Message Frequency

How often should one message/email users?

Once the system is automated, there can be a growing urge among marketers to start sending messages, emails or notifications to the users very frequently. This may result in frustration for the target users. So, there is a need for a feature to learn the optimal message frequency and repetition for each user to increase the metrics like CTR (Click Through Rate), CvR (Conversion Ratio), etc. The recommendation space / duration should be limited in a way that it doesn’t overpower the user’s experience with the application in question.

2. Channel Selection

Which channel(s) best help connect with the user

It is very important for a marketing strategy to be exactly where the users are. With growing innovations in the digital world, users have a large number of available channels to interact with a product (phones and desktops with different operating systems). So, in order to increase the reach and sales, we need to run multichannel campaigns. However, just reaching out to a large number of users is not enough. We need to ensure not only that they receive the messages, but also take the expected action. Hence, we need to have an intelligent product that selects the right channel(s) to reach a particular user based on their past browsing behavior on different devices (given that we have this data already).

3. User Segmentation

Targeting the user by defining and categorizing behavior

User Segmentation refers to grouping of users based on certain properties. The properties may be gender, age, geographical location, interests, buying behavior, channels used, economy, polarity etc. Segmentation is useful for targeting users with relevant offers to upscale sales. But, with the growing size of user data and preferences, it can be very demanding for a human to segment users manually. We know that there are various clustering algorithms which can be used to cluster users in different groups, according to their properties and thus, completely eliminate the human intervention required for this task.

4. Personalization

Playing with product or offer description, positioning, actions, preferences, etc.

Nothing beats a user satisfaction strategy like personalization. It’s very important to interact with users on a personal level in the form of personalized offers, personalized messages or product recommendations based on their previous buying behavior or browsing history. Contextually relevant marketing adds value to a user’s buying experience and can greatly influence their future buying behavior. There are many machine learning algorithms that can take user’s historical features and give personalized recommendations to them. Also, customized offers on birthdays or anniversaries successfully gratify users. The system can also gather purchase points for the user and offer higher discount on products.

5. Natural Language Processing

Sentiment analysis improves personalization

These are some advanced AI techniques which can be very beneficial in marketing strategies. NLP techniques like sentiment analysis can be used to gather the sentiments of the users with respect to various products. Sentiment refers to the emotions of users like sadness, joy, anger, etc. It can be gathered using the user reviews of a product online or from social media posts of users regarding a product. This can provide relevant insights of users regarding a product which can greatly influence future marketing strategies. Since, there are numerous reviews and posts from users. It won’t be feasible to manually extract sentiments out of them when this can be automatically done using AI.

6. User Ranking

Ranking helps make the user experience more intimate

Not all users add the same value to a business. Some are more profitable to a business than others, and hence, require a different level of service. Such users can be identified using a ranking system, which ranks users based on their buying behavior, potential for upselling, etc. Some of the characteristics that can be used to build such an AI system could be revenue, loyalty, service requirements, etc. Once, there is a ranking system in place, all future marketing strategies can be directed successfully.

7. Content Generation

Generate customized content to scale

With the growing number of users buying online, it has become very important to scale up the marketing strategies. AI can not only help in targeting the right users, but also can help in creating personalized content for them. Creating content using AI techniques would become the true form of personalization and would connect with users on a larger scale, thus, improving the connection with the users without manual intervention.

8. Feedback

User feedback to access the performance of the system

No system is complete without a closed feedback loop. Feedback allows us to access the performance of the system and alter them, if necessary. AI techniques can be used to constantly catalogue various user feedback in the form of tagging the posts, reviews, etc. These could also help generate new tags and classify feedback using existing tags.

9. Failure Predication and Recovery

Predict if the system can fail

Every system fails at some point of time. So, it’s very useful to have a fall-back mechanism to which the main system can switch in case of a failure. Also, AI can be used to have a system which can predict if the system is going to fail soon by analyzing the features which led to a failure in the past. Such a mechanism can help in taking measures to prevent a failure in future.

DATABASE SCHEMA

1. Create the table with attributes as shown below, to store the data to be used while implementing the various features of the product.
2. Migrate the existing users and products data, if any, on the database.

HIGH LEVEL SOLUTION

1. User Segmentation

1. Train a clustering algorithm like ‘K-Means Clustering’ using features like age, gender, address, date of birth and revenue present in the user table, sentiment in review table and action type in engagement table.
2. Expose the trained model using a REST API, so that new users could be segmented into one of the existing segments.
3. Store the segmentation information from the model in the ‘segment’ column in the user table.

2. Personalization

1. Train a machine learning model that takes user’s historical features like action type and channel from the engagement table, etc. and outputs a product recommendation and offers specifically tailored for that user.
2. Expose the trained model using a REST API, so that users can be sent new recommendations and personalized offers.

3. Message Frequency

1. Train a regression model that outputs an optimal count of messages and an optimal count of repetitions of each message that should be sent to a user by looking at features like engagement time (time at which the sms/notification arrived) and activity time (time at which the user took an action) in the engagement table.
2. Expose the trained model using a REST API, so that the users can be contacted in an optimal way using the model predictions.

4. Channel Selection

1. Train a classification model that classifies whether a given channel could be useful to market a product to a particular user. Features like ‘channel’ and ‘action type’ in the engagement table can be used for this.
2. Expose the trained model using a REST API, so that users can be sent recommendations on the most relevant channels.

5. User Ranking

1. Train a ranking / regression model that takes in features like revenue in user table and action type, engagement time and action time in the engagement table, and outputs a rank of the user, depicting a profitability metric of that user to the business.
2. Expose the trained model using a REST API, so that the rank of an existing user could be updated or the rank of a new user can be calculated based on an action trigger.

6. Feedback

1. Train an NLP model that classifies user feedback into one of the existing tags, already available or to create a new tag.
2. Expose the trained model using a REST API, check the performance of the various models and retrain them if necessary

7. Failure Prediction and Recovery

1. Train a classification model that predicts whether the deployed system is going to fail in the near future by analyzing the past failures using features like reason, failure time and recovery time in the failure table and the state of the system at the time of the failure.
2. Expose the trained model using a REST API and check at regular time intervals if the system can fail according to the current state of the system.

8. Natural Language Processing

1. Train a classification model that classifies the reviews of the users present in the review table into one of some predetermined sentiments.
2. Expose the trained model using a REST API, so that new user reviews can be classified into a relevant sentiment class.
3. Store the classified sentiment into the Sentiment column in the review table.

9. Content Generation

1. Train an NLP model that generates the personalized marketing content using the customization features explained above.
2. Expose the trained model using a REST API, so that personalized content can be generated before sending a message or notification to a user on relevant channels.