This article provides basic tips and best practices for clients who integrate with Peer39 by implementing URL cache.
Client-side cache maps between URLs and the data that Peer39 provides on each of these URLs.
The general flow follows this procedure.
The request arrives to the client system.
URL is looked up in cache.
If found: Use the data stored in the cache to bid/serve ad. This might depend on the exact response type (see Notes).
If the stored response type is not final (RT=1 or RT=2), forward the URL to Peer39's systems.
If the stored response type is final (RT=0), use the data stored in the cache to bid/serve ad.
If the stored response type is too old, delete it (or mark it as 'to be deleted'), and forward the URL to Peer39's systems.
If not found: Forward the request to Peer39's systems.
As explained in the API Integration documentation, non-final responses (First Look Classification) require the client to keep sending the URL, usually whenever it is requested using the exchange/ad call, until a final response is available. The client can elect whether to store and/or serve First Look classifications.
The cache should also include a purging mechanism for little-used or old entries. Peer39 purges all URLs after 24 hours, and we recommend clients to do the same. Benefits of purging include:
Saving memory resources
Improving cache performance
Having the freshest data on the most recent URLs
In general, the cache requires many more seek operations than store operations. The choice of data structure should consider this performance issue.
The size of the cache, and the appropriate data structure and hardware requirements, can be estimated by analyzing the traffic.
A major disadvantage when implementing a cache is data synchronization across machines and/or data centers. We recommend that every effort is made to make the cache fit in a single machine, preferably the same one that serves/bids.
If using the data in more than one form, the exact format of data saved in the cache should be decided. We recommend saving the finalized results in the cache, even if this requires more memory, rather than processing large amounts of data online.
To allow fast integration, the system should provide a simple way of extracting the cache contents, and getting insight into the timeline of requests sent and replies received from Peer39.