Multi-level cache in microservice architecture should be designed like this!

Thisisacommunitythatmaybeusefultoyou

One-to-onecommunication/interviewbrochure/resumeoptimization/jobsearchquestions,welcometojointhe”YudaoRapidDevelopmentPlatform“KnowledgePlanet.ThefollowingissomeinformationprovidedbyPlanet:

  • “ProjectPractice(Video)”:Learnfrombooks,“practice”frompastevents

  • “InternetHighFrequencyInterviewQuestions”:Studyingwithyourresume,springblossoms

  • “ArchitecturexSystemDesign”:Overcomingdifficultiesandmasteringhigh-frequencyinterviewscenarioquestions

  • “AdvancingJavaLearningGuide”:systematiclearning,themainstreamtechnologystackoftheInternet

  • “Must-readJavaSourceCodeColumn”:Knowwhatitisandwhyitisso

Thisisanopensourceprojectthatmaybeusefultoyou

DomesticStarisa100,000+opensourceproject.Thefront-endincludesmanagementbackend+WeChatapplet,andtheback-endsupportsmonomerandmicroservicearchitecture.

FunctionscoverRBACpermissions,SaaSmulti-tenancy,datapermissions,mall,payment,workflow,large-screenreports,WeChatpublicaccount,etc.:

  • Bootaddress:https://gitee.com/zhijiantianya/ruoyi-vue-pro

  • Cloudaddress:https://gitee.com/zhijiantianya/yudao-cloud

  • Videotutorial:https://doc.iocoder.cn

Source:Internet

  • Multi-levelcachedesigninmicroservicearchitecture

  • clientcache

  • Applicationlayercaching

    • CDNContentDeliveryNetwork

    • Nginxcachemanagement

  • Servicelayercaching

    • In-processcache

    • Distributedcacheservice

    • Ensurecacheconsistency

  • summary

Todaywewilltalkaboutthetopicofcachingandseehowtodesignaneffectivemulti-levelcachearchitectureinamicroserviceenvironment.Itmainlyinvolvesthreeaspects:

  • Client-sidecachingforwebapplications;

  • Applicationlayerstaticresourcecaching;

  • Servicelayermulti-levelcaching.

First,let’sexplainthemulti-levelcachedesignofmicroservicearchitecture.

Multi-levelcachedesigninmicroservicearchitecture

Whenitcomestocaching,everysoftwareengineermustbefamiliarwithit.Itisthemostdirectwaytoimproveperformanceincurrentarchitecturedesign.Herewegiveanexample:

Rediscache

AssumethattheapplicationstoresrawdatainaMySQLdatabase.Asweallknow,theMySQLdatabasestoresdataontheharddisktopreventlossduetopowerfailure.However,duetothephysicaldesignoftheharddisk,eventhebest-performingenterprise-classSSDharddiskisanorderofmagnitudeworsethantheIOlevelofhigh-speeddevicessuchasmemory.Internetapplicationsrepresentedbye-commercecompaniessuchasTaobaoandJD.comaretypicalscenariosof”morereadingandlesswriting”.Therefore,weneedtoseparatethereadingandwritingofdatainthedesign.Whendataisloaded,itisdirectlyplacedonthediskforprocessing,whilemorethan90%ofthedatareadingoperationsareextractedfromthein-memoryNoSQLdatabaserepresentedbyRedis,andthehighthroughputofthememoryisusedtocompletethedataextractioninstantly.TheroleofRedishereiswhatweoftensaycache.

Ofcourse,cachingisnotjustaformofreplacingharddiskswithmemory.Inadistributedarchitecture,cachinghasitsowndesignateachlayer.Let’sexplainitthroughthemulti-levelcachearchitecturediagramofthismicroserviceasthemainline.

OverviewofXcachemulti-levelcachearchitecture

Thisdiagramcontainsfourlayersfromtoptobottom:Client,applicationlayer,servicelayeranddatalayer.

Backendmanagementsystem+userappletimplementedbasedonSpringBoot+MyBatisPlus+Vue&Element,supportingRBACdynamicpermissions,multi-tenancy,datapermissions,workflow,three-partylogin,payment,SMS,mallandotherfunctions

  • Projectaddress:https://github.com/YunaiV/ruoyi-vue-pro

  • Videotutorial:https://doc.iocoder.cn/video/

Clientcache

TheXmallclientisabrowser.Atthebrowserlevel,wemainlycachestaticresourcessuchasimages,CSS,JS,andfontsinHTML.

browsercache

Let’staketheBaiduLogoimageasanexample.BaiducontrolsthevalidityperiodofthestaticimagethroughtheExpiresresponseheaderinHTTP.Expiresrepresentsexpirationtime.ThecurrentBaiduLogoexpirationtimeisFebruary8,2031,9:26:31.Duringthisperiodoftime,thebrowserwillcachetheimagelocallyasafile.Whenyouvisitagain,youwillseetheprompt”fromdiskcache”.Atthistime,thebrowserwillnolongermakeanactualrequesttotheserverandwillreaditdirectlyfromthelocalcomputer.Getcachedimages.BysettingExpiresonthebrowserside,thebandwidthlosscausedbyrepeatedrequestsforstaticresourcescanbegreatlyreduced.Thisisabasicandimportantsettinginhigh-concurrencywebapplications.

Backendmanagementsystem+userappletimplementedbasedonSpringCloudAlibaba+Gateway+Nacos+RocketMQ+Vue&Element,supportingRBACdynamicpermissions,multi-tenancy,datapermissions,workflow,three-partylogin,payment,SMS,mallandotherfunctions

  • Projectaddress:https://github.com/YunaiV/yudao-cloud

  • Videotutorial:https://doc.iocoder.cn/video/

Applicationlayercache

SowhereisExpiresset?Forthebrowser,itisjustaclientandisonlyresponsibleforreadingtheExpiresresponseheader.Expiresmustbesetattheapplicationlayer,thatis,CDNandNginx.

CDNContentDeliveryNetwork

ThefullnameofCDNisContentDeliveryNetwork,whichisthecontentdistributionnetwork.ItisthemaintechnicalmeansforthedistributionofstaticresourcesontheInternet.

CDNContentDeliveryNetwork

Chinaisavastcountry,withthousandsofkilometersfromBeijingtoShanghai.IfalargenumberofShanghaiuserswanttoaccesstheresourcesofserversinBeijingthousandsofmilesawayatthesametime,suchalongcommunicationwillinevitablybringhighdelaysandmoreuncontrollablefactorsaffectingdatatransmission.IfthereissomemechanismthatallowsstaticfilesinBeijingtobecachedonserversinShanghai,Shanghaiuserscanautomaticallyaccessthenearestservertoobtainresources.Thiscangreatlyreducenetworklatencyandimprovesystemavailability.ThedistributedcachingtechnologyjustmentionedistheCDN(contentdistributionnetwork)weoftenmention.

Forwide-areaInternetapplications,CDNisalmostanecessaryinfrastructure,whicheffectivelysolvestheproblemsofconcentratedbandwidthoccupationanddatadistribution.Staticresourcessuchasimages,audioandvideo,CSS,andJSinwebpagescanbeobtainedfromthenearestCDNserver.

ThecoreofCDNtechnologyis”smartDNS”.SmartDNSwillautomaticallydeterminethenearestaccessCDNnodebasedontheuser’sIPaddress.Let’stakethefollowingfigureasanexample:

CDNexecutionprocess

AssumethataShanghaiuser’sbrowserwantstoaccessthebanner.jpgfileintheadvertisingspaceonthemall’shomepage.ThebrowserautomaticallyforwardstherequesttotheCDNserverpreparedbythemallinShanghaithroughtheintelligentDNSserviceprovidedbytheserviceprovider.AftertheShanghaiCDNreceivestherequestFirstcheckwhetherthebanner.jpghasbeencachedonthismachine.Ifthefilealreadyexists,theimagedatawillbereturneddirectlytotheclient;ifithasnotbeencached,returntothesourcedatanodeinBeijing,extractthebanner.jpgfileandcacheittoShanghaiserver,andfinallytheShanghaiCDNnodereturnsthelocalbanner.jpgtotheclient.Forbanner.jpg,iftheShanghaiCDNnodehascachedthefileafterthefirstaccess,thenallsubsequentaccesseswithinthecachevalidityperiodwillbedirectlyprovidedbyShanghaiCDN.Similarly,mallapplicationscansetupCDNnodesinimportantcities,sothatrequestsoriginallysenttotheBeijingserveraredistributedtotheCDNnodes,whichdirectlyreducesthebandwidthpressureontheBeijingcomputerroom.

InInternetapplications,becauseCDNinvolvesmulti-regionandmulti-nodenetworking,theinitialinvestmentcostisrelativelyhigh.Moresmallandmedium-sizedsoftwarecompaniesusuallychooseCDNservicesprovidedbymajormanufacturerssuchasAlibabaCloudandTencentCloud,andpayondemand.Reducehardwarecosts.TheseserviceproviderswillprovideadditionalcapabilitiestoCDN.Forexample,AlibabaCloudandTencentCloudCDNnotonlycachefiles,butalsoprovideamanagementbackendthatcanassignadditionalresponseheaderstoresponses.Asshownbelow,intheAlibabaCloudCDNbackend,anadditionalCache-Controlresponseheaderissettorepresentacachevalidityperiodof1hour.HerewewouldliketomentionthedifferencebetweenExpiresandCache-Control.Expiresspecifiesthecacheexpirationataspecificpointintime,whileCache-Controlrepresentsthevalidityperiodofthecache.ExpiressetsthetimeandCache-Controlsetstheduration.Differentresponseheaderscanbeusedaccordingtodifferentbusinessscenarios.

AlibabaCloudcustomresponseheader

Nginxcachemanagement

AftertalkingaboutCDN,let’stalkaboutNginx.Nginxisanopensource,cross-platform,high-performancewebserver.Ithastheadvantagesofhighperformance,goodstability,simpleconfiguration,structuredmodules,andlowresourceconsumption.Italsosupportsreverseproxy,loadbalancing,andcachingfunctions.Nginxisafrequentvisitorinwebapplicationarchitecture.Forexample,theback-endTomcatclustercanprovidehigh-availabilityfeaturesforapplicationsbyaddingNginxfront-endforsoftloadbalancing.

Nginxproxyserver

InInternetapplications,usersaredistributedacrossthecountryandhavehighresourceresponsespeedandbandwidthrequirements,soitisverynecessarytodeployaCDN.However,inmoreenterpriseapplications,infact,mostenterpriseusersaredistributedindesignatedofficeareasorrelativelyfixedplaces,andtherearerelativelyfewconcurrentusers.Infact,thereisnoneedtodeployaheavyweightsolutionsuchasCDN..Inthearchitecture,youonlyneedtodeployanNginxserveranduseNginx’sbuilt-instaticresourcecachingandcompressionfunctionstomeetmostenterpriseapplicationscenarios.

Nginxcomeswiththefunctionofcachingstaticresourcessuchasimages,CSS,andJSinback-endapplications.WeonlyneedtoaddthefollowingfragmenttoNginx’scoreconfigurationnginx.conftocachethestaticresourcesoftheback-end.KeyconfigurationIhavemadeannotationsandstudentscanusethemdirectly.

#Setcachedirectory
#levelsmeanssavingcachefiles(staticresourcescss,js)intheformof1:2,whichisatwo-leveldirectory.
#keys_zonedefinesthenameofthecacheandtheuseofmemory.Thenameisbabytun-cache,and100mofswapspacestartsinthememory.
#inactive=7dIfacachefilehasnotbeenaccessedformorethan7days,deleteit
#max_size=20g;meansthatthemaximumsettingfoldercannotexceed20g.Afterexceedingthelimit,thecachefilewiththelowestaccessfrequency(hitrate)willbeautomaticallydeleted.
proxy_cache_pathd:/nginx-cachelevels=1:2keys_zone=babytun-cache:100minactive=7dmax_size=20g;

#Configuretheweightedloadbalancingstrategyofthexmallbackendserver
upstreamxmall{
server192.168.31.181weight=5max_fails=1fail_timeout=3s;
server192.168.31.182weight=2;
server192.168.31.183weight=1;
server192.168.31.184weight=2;
}

server{
#nginxprovideswebservicesthroughport80
listen80;
#Enablestaticresourcecaching
#UseregularexpressionstomatchURLs.Ifthematchissuccessful,theinternallogicwillbeexecuted.
#~*meansURLmatchingisnotcasesensitive
location~*\.(gif|jpg|css|png|js|woff|html)(.*){
#Configureproxyforwardingrules
proxy_passhttp://xmall;
proxy_set_headerHost$host;
proxy_set_headerX-Forwarded-For$proxy_add_x_forwarded_for;
proxy_cachexmall-cache;
#Ifthestaticresourceresponsestatuscodeis200(success)or302(temporaryredirection),thecachefileisvalidfor1day.
proxy_cache_valid20030224h;
#301(PermanentRedirect)Cachekeptfor5days
proxy_cache_valid3015d;
#Othercases
proxy_cache_validany5m;
#Setthebrowsercacheexpirationtimeto90days
expires90d;
}



#Usexmallserverpoolforbackendprocessing

location/{
proxy_passhttp://xmall;
proxy_set_headerHost$host;
proxy_set_headerX-Forwarded-For$proxy_add_x_forwarded_for;
}
}

Afteraddingtheaboveconfiguration,everytimeanewstaticfileintheapplicationisaccessedthroughNginx,acachefilewillbegeneratedinthecachedirectoryoftheNginxservice.Duringthecachevalidityperiod,therequestforthestaticresourcewillnolongerbesenttotheback-endserver,butdirectlyfromNginxreadsthelocalcacheandreturnsit.

Nginxlocalcache

Servicelayercache

Inthepast,bothCDNandNginxcachedstaticresourcefilesinwebapplications.Butback-endapplicationsandservicesaremoreaboutaccessinterfacesanddata.Howcanweusecachingtechnologytooptimizeperformancefortheseobjects?Cachingforback-endapplicationsandservicescanbedividedintoin-processcachinganddistributedcachingservicesbasedondeploymentmethods.

In-processcache

Theso-calledin-processcacheisamemoryspaceopenedintheapplication.Dataisloadedintothismemoryduringruntimetoimprovetheaccessspeedoftheprogramthroughthelow-latencyandhigh-throughputcharacteristicsoflocalmemory.In-processcachingiswidelyusedinmanyJavaframeworks.Forexample,Hibernate,thefirst-andsecond-levelcacheoftheMybatisframework,andthepagecacheofSpringMVCareallclassicapplicationscenariosofin-processcaching.Thesein-processcachesalsohavemanyapplicationsinJava.Excellentopensourceimplementations,suchasEhCacheandCaffeine,arerepresentativeproducts.

Distributedcacheservice

Theoppositeofin-processisadistributedcacheservicethatneedstobedeployedindependently.Themostcommonlyusedoneisbasedonanin-memoryNoSQLdatabasesuchasRedis,whichcentrallycachesapplicationdataintheoverallarchitecture.

Whendesigningthearchitecture,manynewarchitectssubconsciouslythinkthataddingaRedisdistributedcacheserverisenoughwhentheyhearaboutcaching.Infact,thisisaone-sidedapproach.Whendesigningthecachearchitecture,youmustperformlevel-by-levelaccessintheorderfromnearesttofarandfromfasttoslow.Supposethatwhenane-commercecompanyconductsproductflashsaleactivities,ifthereisnolocalcache,allhotspotdataofproducts,orders,andlogisticsarestoredintheRedisserver.Eachtimeanorderiscompleted,severaladditionalnetworkcommunicationsarerequired.ThenetworkcommunicationitselfisTheremaybecommunicationfailuresduetovariousreasons.Evenifyoucanensurethatthenetworkis100%available,theRedisclusterbearstheaccesspressurefromallexternalapplications.OncethebursttrafficexceedstheloadlimitofRedis,theentirearchitectureisatriskofcollapse.

Rediscacheservicecluster

Therefore,multi-levelcachingmustalsobedesignedontheJavaapplicationside.Wecombinein-processcachingwithdistributedcachingservicestoeffectivelyshareapplicationpressure.AttheJavaapplicationlevel,onlywhentheEhCachecachedoesnotexist,gototheRedisdistributedcachetoobtainit.IfRedisdoesnothavethisdata,gotothedatabasetoquery.Afterthedataqueryissuccessful,RedisandEhCahcewillbedouble-writtenandupdatedatthesametime.Inthisway,thenexttimetheJavaapplicationqueriesthesamedata,itwillbedirectlyextractedfromthelocalEhCachecache,nonewnetworkcommunicationwillbegenerated,andtheapplicationqueryperformancewillbesignificantlyimproved.

Multi-levelcachedesign

Guaranteecacheconsistency

Butnothingisperfect.Whenmulti-levelcacheisintroduced,wewillencounterthechallengeofcachedataconsistency.Thefollowingfigureisanexample:

Cacheconsistencyissues

Weallknowthatdatabasewriteoperationsdonotgothroughthecache.Assumethatproductserviceinstance1adjuststhepriceofproductNo.1to80yuan.Thiswillraiseanewquestion:Howtoactivelypushdatachangemessagestoapplicationstoensurethattheycanalsoupdatethecachesynchronously?

Ibelieveyoualreadyhavetheansweratthispoint.Yes,weneedtointroduceMQmessagequeueintothecurrentarchitectureanduseRocketMQ’sactivepushfunctiontoinitiatechangenotificationstootherserviceinstancesandtheRediscacheservice.

SolvetheproblemofensuringmessageconsistencythroughRocketMQ

Asshowninthefigureabove,afterproductserviceinstance1adjuststhepriceoftheproduct,itactivelysendsachangemessagetoRocketMQBroker.TheBrokerpushesthechangeinformationtootherinstancesandtheRediscluster.Afterreceivingthechangemessage,theseserviceinstancesarefirstdeletedfromthecache.Expirethecacheandcreatenewdatatoensurethatthedataofeachinstanceisconsistent.

Whenyouseethis,youwillfindthatthereisnoultimatesolutionforcaching.Althoughthemulti-levelcachedesignbringsbetterapplicationperformance,MQmustbeintroducedforcacheconsistency,whichincreasesthecomplexityofthearchitecture.Sohowtochoosebetweenmulti-levelcachedesigns?Inmyopinion,therearethreesituationsthatareparticularlysuitableforintroducingmulti-levelcaching.

Inthefirstcase,thecacheddataisstable.Informationsuchaspostalcodes,geographicalblocks,andarchivedhistoricaldataaresuitableforreducingthepressureonRedisandthedatabasethroughmulti-levelcaching.

Inthesecondcase,extremelyhighconcurrencyscenariosmayoccurinstantaneously.Forexample,duringtheSpringFestivaltravelticketpurchase,Double11midnightflashsales,stockmarketopeningtransactions,etc.,theinstantaneoustrafficpeakmaypenetratetheRediscacheandcauseatrafficavalanche.Atthistime,itisverynecessarytousethepreheatedin-processcachetodistributetrafficandreduceback-endpressure.

Inthethirdcase,datainconsistencyisallowedtoacertainextent.Forexample,inablogplatform,youmodifynon-criticalinformationsuchasself-introduction.Atthistime,thecacheinconsistencyofothernodesintheapplicationclusterwillnothaveaseriousimpact.Forthissituation,weuseT+1tohandleitattheendoftheday.Itisenoughtoensurethatthecacheiseventuallyconsistent.

TheabovearethreescenariosthatIhavesummarizedthataresuitableformulti-levelcachingintheservicelayer.Ofcourse,ifyourapplicationconcurrencyisnotlargeandtheRedisdistributedcacheclustercanfullymeettheapplicationperformancerequirementsinthenext1to2years,thenthereisnoneedtodesignamulti-levelcache.Wemustflexiblyadjustthearchitectureaccordingtobusinesscharacteristics.

Summary

Todayweintroducedthecachingdesignandsolutionsforeachlayerfromtheclienttotheservicelayerundertheapplicationmicroservicearchitecture,andexplainedfromthebrowser’sExpiresresponseheadertothestaticresourcecacheofCDNandNginx,andthentotheservicelayerfordataThemulti-levelcachegivesyouanoverallunderstandingofcachinginmicroservicearchitecture.

Welcometojoinmyknowledgeplanetandcomprehensivelyimproveyourtechnicalcapabilities.

Tojoin,“Longpress”or“Scan”theQRcodebelow:

Planet’scontentincludes:projectpractice,interviewsandrecruitment,sourcecodeanalysis,andlearningroutes.

Ifthearticleishelpful,pleasereaditandforwardit.
Thankyouforyoursupport(*^__^*)

Theknowledgepointsofthearticlematchtheofficialknowledgearchives,andyoucanfurtherlearnrelevantknowledge.Cloudnativeentry-levelskillstreeServicegrid(istio)ServiceMeshintroduction16803peoplearelearningthesystem