Backup plan File type Frequency Tools and Storage Sync / Copy Encryption dotfiles daily git + github Sync N /etc files daily etckeeper + git + github Sync N scripts daily git + github Sync N documents daily rclone + onedrive / google drive Sync N pictures weekly rclone + dropbox / pcloud Copy Y private/sensitive data weekly rclone + dropbox / pcloud Copy Y learning materials (courses + languages) weekly rclone + alist + baidunetdisk / aliyundrive Copy N ebooks weekly rclone + alist + baidunetdisk / aliyundrive Copy N music weekly rclone + alist + baidunetdisk / aliyundrive Copy N movie weekly rclone + alist + baidunetdisk / aliyundrive Copy N applications monthly rclone + alist + baidunetdisk / aliyundrive Copy N system backup monthly restic + Hard disk drive Sync Y * – syncthing Sync N Configuration files For tracking changes and backing up configuration files I use etckeeper for /etc and chezmoi (really cool piece of software) for my dotfiles.
Performance Tuning
SQL Performance Tuning Add missing indexes
General rules of thumb:
Put index on columns that are being used in where clauses Put index on columns that you use to join on. Try not to use more than 4-5 indexes on columns in the same table. Don’t use indexes on columns with only few different values. The table would consume more space with the index and preform slower on insertion, but you won’t get significant better performance while querying General concepts:
System Design
Reference Well-Architected
Authentication And Authorization
Access token JWT (JSON Web Token) What is JSON Web Token? JSON Web Token (JWT) is an open standard that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. This information can be verified and trusted because it is digitally signed. JWTs can be signed using a secret (with the HMAC algorithm) or a public/private key pair using RSA or ECDSA.
For example, a server could generate a token that has the claim “logged in as administrator” and provide that to a client.
microservices with go
Creating a Service Health Checking Authentication Access token JSON Web Token Service Discovery Load-balancing Centralized configuration Spring Cloud config Viper Messaging Request/response - a service sends a request message to a recipient and expects to receive a reply message promptly Notifications - a sender sends a message a recipient but does not expect a reply. Nor is one sent. Request/asynchronous response - a service sends a request message to a
Use API gateways in microservices
What is an API gateway? An API gateway sits between clients and services. It acts as a reverse proxy, routing requests from clients to services. It may also perform various cross-cutting tasks such as authentication, SSL termination, and rate limiting.
If you don’t deploy a gateway, clients must send requests directly to front-end services. However, there are some potential problems with exposing services directly to clients:
It can result in complex client code.
HTTP Cookie Conversion
cookie string to dictionary 1 from http.cookies import SimpleCookie 1 rawdata = 'sa-user-id=s%253A0-ef03aab2-d5bd-5b8c-50cd-a395819c57e1.a6YKYr5eOTh3f3vgcQJ3jVz540XviiIfrLMSZ8gSTmQ; sa-user-id-v2=s%253A7wOqstW9W4xQzaOVgZxX4bcLJAo.AaBzjGi%252FSqBzVUBWHWNyhlZugJhn1zViN4PYEycbpSE' 1 2 3 4 5 cookie = SimpleCookie() cookie.load(rawdata) cookies = {} for key, morsel in cookie.items(): cookies[key] = morsel.value 1 print(cookies) cookie dictionary to string 1 cookie_string = "; ".join([str(x)+"="+str(y) for x, y in cookies.items()])
Encryption
Two Main Types Of Keys Symmetric-key In symmetric-key schemes, the encryption and decryption keys are the same. Communicating parties must have the same key in order to achieve secure communication.
The requirement that both parties have access to the secret key is one of the main drawbacks of symmetric-key encryption, in comparison to public-key encryption.
However, symmetric-key encryption algorithms are usually better for bulk encryption. Asymmetric-key encryption is often used to exchange the secret key for symmetric-key encryption.
Data engineer roadmap
Data Engineer in 2021 CS fundamentals Basic terminal usage [general recommendation] Data structures & algorithms [general recommendation] APIs [general recommendation] REST [general recommendation] Structured vs unstructured data [general recommendation] Serialisation Linux [general recommendation] CLI Vim Shell scripting Cronjobs How does the computer work? [general recommendation] How does the Internet work? [general recommendation] Git — Version control [general recommendation] Math & statistics basics [general recommendation] Note: Git is used for tracking changes in source code and coordinating work among programmers.
How To Use the STAR Interview Response Technique
What is the STAR interview method? The STAR interview method is a technique you can use to prepare for behavioral and situational interview questions. STAR stands for: situation, task, action and result. This method will help you prepare clear and concise responses using real-life examples. How does the STAR method work? The STAR method helps you create an easy-to-follow story with a clear conflict and resolution. Here
SWOT Analysis Guide (With Examples)
A SWOT analysis is a way to evaluate strengths, weaknesses, opportunities and threats. You might perform this analysis for a product, team, organization, leadership or other entities. SWOT analyses are used in many business environments to gain a better understanding of how to plan for the future.
In this article, we discuss what a SWOT analysis is in further detail, why they are used and how to perform a SWOT analysis.
business-analyst
How would you work with a difficult stakeholder? As a business analyst, you will likely deal with many different personalities occupying various positions. Situational questions like this one measure your problem-solving skills, communication skills and ability to resolve difficult situations. This question assesses whether you can successfully navigate interactions with many different stakeholders. Provide a direct answer and explain a related challenge you faced in past work. You can use
business-analyst-interview-questions
你会如何与困难的利益相关者合作? 作为业务分析师,您可能会与担任不同职位的许多不同性格的人打交道。像这样的情境问题衡量你解决问题的能力、沟通技
How To Prepare for a Behavioral Interview
What is a Behavioral Interview? A behavioral interview is a popular interview technique employers use to assess job candidates based on their past behavior. For example, instead of asking hypothetical questions like, “How would you react if you were under a lot of pressure at work?” they would instead ask situational questions like, “Describe
Product Manager Interview Questions
Common interview questions Self-Awareness 请介绍一下下你自己 你好,我是郑烨,首先很感谢您今天抽出时间对我进行面试。 首先,我从毕业以后就从事编程开发的工作,我在软件开发方面拥有
distributed systems
Java分布式应用如何入门以及有哪些资料? 怎样才是一个基本水平的java程序员?
load balance
Load balancing
Distributes client requests or network load efficiently across multiple servers Ensures high availability and reliability by sending requests only to servers that are online Provides the flexibility to add or subtract servers as demand dictates Load Balancing Algorithms
summarization
https://github.com/huggingface/transformers/tree/master/examples/seq2seq https://github.com/tensorflow/models/tree/master/research/textsum https://github.com/atulkum/pointer_summarizer https://github.com/vrindav/just-news-it https://github.com/pytorch/fairseq https://github.com/abisee/pointer-generator
translation
For all translation problems, we suggest to try the Transformer model.
code quality gate
sourcemonitor simian
coverity fotify klocwork
Sonar checkstyle-idea findbugs-idea PMDPlugin Save Actions