web crawler interview questions

Scrapy 爬取数据的时候遇到了什么反爬虫情况?你了解有哪些反爬虫机制? 初级: 网络基础:cookie, session, https, headers常用的字段,代理使用 Pytho

web interview questions

什么是正向代理和反向代理? 正向代理位于客户端和原始服务器之间,代表客户端向原始服务器发送请求。客户端向正向代理发出请求,然后由正向代理将请求

达观数据面经

达观数据面经 一轮笔试 二分查找 (Binary Search) 1 2 3 4 5 6 7 8 9 10 11 12 class Solution: def search(self, nums: List[int], target: int) -> int: left, right = 0, len(nums) - 1 while left <= right: pivot = left + (right - left) // 2 if nums[pivot] == target: return pivot if target < nums[pivot]: right =

Deploying machine learning models in production

Reference 训练好的深度学习模型是怎么部署的? PyTorch模型的加速及部署 ahkarami/Deep-Learning-in-Production Accelerating Inference In TF-TRT User Guide tensorflow/tensorrt Tensorrt API Reference Containers: nvidia:tensorflow Model inference using TensorFlow and TensorRT Speed up TensorFlow Inference on GPUs with TensorRT TF20-TF-TRT-inference-from-Keras-saved-model.ipynb

Network Programming

第1章 简介 异步回调 TCP/IP四层体系结构(用户进程、内核中的协议栈、以太网) 局域网 广域网 客户端的简单实现 if ((sockfd = socket(AF_INET, SOCK_STREAM, 0)) < 0); bzero (&serveraddr, sizeof(serveraddr); serveraddr.sin_family = AF_INET; serveraddr.sin_port = htons(port);

TOTP: Time-based one-time password

1 2 TOTP = HOTP(K, T) HOTP(K,C) = Truncate(HMAC-SHA-1(K,C)) T:T = (Current Unix time - T0) / X, T0 = 0,X = 30 K: HOTP: Time-based one-time password (TOTP) algorithm (RFC 6238) implies that an one-time password (OTP) is a product of two parameters encrypted together. These are a common value, which is a shared secret key; and a variable,

Trustworthiness

High availability High availability cluster redundant, eliminate single points do failure Heartbeat, monitor the healthy and status of each node in the cluster. node configuration: active / active active / passive N + 1 failover strategies High reliability availability service high availability Database high availability backup backup everyday Data recovery disaster recovery planning privacy data masking Security Resilience Safety

Unix Programming

进程间通信IPC: TCP socket(只用这个):好处是进程关闭了能够被感知,坏处是太慢了。 进程信号(消息队列): 共享内存文件: 管道: 进程间堆和

web security

CS 253 Web Security https://zhuanlan.zhihu.com/p/30027739 https://www.cs.fsu.edu/~redwood/OffensiveComputerSecurity/lectures.html

web security

网站用户密码保存 明文保存 明文hash后保存,如md5 MD5+Salt方式,这个salt可以随机 知乎使用了Bcrypy(好像)加密 XSRF和X

Common Service

APM(Application Performance Management) ADS(Application Deploy Service) WAS(WebSphere Application Server) Dynatrace Pinpoint UEM用户体验运营 ADS RDS API Gateway: Kong/Kong Centralized config server: Apollo Nacos Spring Cloud Config Go Microservices, Part 8: Centralized Configuration With Viper and Spring Cloud Config spring-cloud-alibaba APM: Pinpoint CodeHub: GitLab Stream Compute Service Software-define Storage Distributed Storage Ceph gluster

Getting Started with Microservice Development Using Spring Boot

Introduction to Microservices Benefits Each component in the platform can be delivered and upgraded separately. Each component in the platform can also be scaled out to multiple servers independently Challenges Many small components that use synchronous communication can cause a chain of failure problem, especially under high load. Keeping the configuration consistent and up to date in all the instances Monitoring the state of the platform in terms of latency

Leveraging Spring Cloud to Manage Microservices

Introduction to Spring Cloud Design pattern Current component Replaced by Software component Service discovery Netflix Eureka & Netflix Ribbon Netflix Eureka and Spring Cloud load balancer Edge server Netflix Zuul Spring Cloud Gateway and Spring Security OAuth Centralized configuration Spring Cloud Configuration Server Circuit breaker Netflix Hystrix Resilience4j Distributed tracing Spring Cloud Sleuth and Zipkin Adding Service Discovery Using Netflix Eureka and Ribbon Using Spring Cloud Gateway to Hide Microservices Behind an Edge Server Securing Access to APIs Centralized Configuration Improving Resilience Using Resilience4j Understanding Distributed Tracing