Yesterday, I said the method of evaluation seems like so limited, so I’m trying my best to search the new ways to do the evaluation.
At this time, I find the manifold learning is also a good way to achieve my goal. It could reduce the dimension of high dimension data, the index system is always in high dimension, so if we could reduce the dimension of index system, then we’ll get the evaluation result.
Actually, manifold is not the shape, it is a space, it means we could embed a 2 dimension manifold in 3 dimension space, then this low dimension manifold would homeomorphism to the Euclidean Space with 2 dimension, for example, in the surface of sphere, it would be expressed with polar coordinates $\mathbb{R}^n$. $$ x = r sin\theta cos \phi \\ y = r sin \theta sin \phi \\ z = r cos \theta $$ You could see that the 3 dimension is controled by 2 variables
If we defined a $d$ dimension manifold in Euclidean Space $\mathbb{R}^d$, $f:Y \rightarrow \mathbb{R}^d$ is a smooth embedding. As for $D>d$ the order of manifold learning is reconstructing the $Y$ and $f$ according to the observed variable $x_i$ in $\mathbb{R}^D$. The hedden data $y_i$ is generated by $Y$ randomly. it could produce the observed data by embedding the $f$, it means $x_i = f(y_i)$.
MDS(MultiDimensional Scaling) is a custom method of reducing the dimension, the target of MDS is saving the dissimilarity in high dimension, if $x \in \mathbb{R}^D$, $x^{\prime} \in \mathbb{R}^d$, $D>d$, the target function is this. $$ \operatorname{min} \sum_{i,j}dist(x_i,x_j)dist(x_i^{\prime},x_j^{\prime}) $$
Then we could calculate the distance matrix $D = [d_{i,j}]=[dist(x_i,x_j)]$, then we could construct matrix $A = [a_{i,j}] = [ \frac{1}{2} d_{i,j}^2]$, after the central correction, we get the matrix $B = JDJ$, $J=I\frac{1}{n}O$, among this, the $I$ is the unit matrix with $n \times n$, $O$ is a $n \times n$ matrix which is full of value 1. Then we could get the feature vector, $e_1,e_2,\dots,e_m$, and the feature value is $\lambda_1,\lambda_2,\dots.\lambda_m$, as last we could reconstruct $X^{\prime} = E_k \boldsymbol{\Lambda}_k^{\frac{1}{2}}$, the $\boldsymbol{\Lambda}_k$ is the diagonal matrix which is constructed with the max value of feature value. $E_k$ is a matrix which is made by $k$ feature vector.
LE(Laplacian Eigenmap) is a method which means the closed points in high dimension will also be closed in low dimension.
If we need to do this method, we could get the adjacency graph, according to the Hear Kernel, the $W_{ij} = e^{\frac{x_i  x_j^2}{t}}$.
Then we could start to do the feature map. $$ Lf = \lambda Df $$ In this equation, $D_{ii} = \sum_j W_{ji}$, $L=DW$ is the laplacian matrix.
The answer of this equation is $f_0,\dots,f_{k1}$, and $0=\lambda_0 \leq \lambda_1 \leq \lambda_{k1}$, the feature in low dimension is $f_1, f_2, \dots, f_d$.
At last the manifold learning is a excellent tool in reducing the high dimension, but we could pay attention to the loss of this method. I also have a thought in this method, if we could combine the result of each manifold learning method?
]]>Recently, I am studying the interesting model of evaluation model, but I saw a lot of information about PCA or entropy weight method. Of course, these methods is really excellent, but they are so old, and as I know, the entropy is depended on the amount of the variable information. To some extend, it’s not so fit for our requirements, so I has found a rare method of this problem, which called Projection Pursuit
Actually, the core of this method is such a easy problem. We just need to project the data from high dimensions onto the single dimension, then we could complete the evaluation process. But this method turn the evaluation to an optimization problem.
First of all, we need to normalize the data and make the value of variable become the range of [0,1].
$$ X^{\prime} = \frac{X  X_{min}}{X_{max}X_{min}} $$
Then we could construct the projection function, and the original data could times direction vector then project them onto the low dimension and the projection funcion could be there. $$ z_i = \sum_{j=1}^m a_j x_{ij} $$ The $a$ is the direction vector, and the $x_{ij}$ is the index value.
After that, we could get the target function and the subject conditions, if the X has $m$ dimensions the whole equation would be this.
$$ \begin{cases} \operatorname{max} Q(a) = \sigma \times D_z \\ \operatorname{s.t.} \sum_{j=1}^m a_j^2 = 1, a_j \in [0,1] \end{cases} $$
In this equation, the $\sigma$ and $D_z$ is this.
$$ \sigma = \sqrt{\frac{\sum_{i=1}^n (z_i  z_{mean})^2}{n1} }\\ D_z = \sum_{i=1}^n \sum_{j=1}^n (Rr_{ij}) o (R  r_{ij}) $$
The $\sigma$ is the standard deviation of $z_i$ and the $D_z$ is the local density, $R$ is the window radius for local density, $r_{ij}$ is the distance between sample, the $o{\tau}$ is the unit step function, when $\tau \ge 0$, the value of it is 1, if $\tau < 0$, the value is 0.
Then we could use a lot of optimization to solve this problem, and we could get the result by MATLAB easily.
Here I provide a function which is made by MATLAB.


Finally, we could get the value of goal function, and I have used MultiAgent Genetic Algorithm to solve this problem.
Then we could know the weight of each variable.
At last, even though it seems that some people don’t pay attention to this method of evaluation, but I think it’s a great way to combine the optimazation with the evaluation.
]]>As we all know, the SDE(Stochastic Differential Equation) is also an important tool in finance, like the famous BlackScholes Equation is also derived by the SDE, and its root is the equation we’ll talk about today.
The GBM could be like this form. $$ \mathrm{d} S_t = \mu S_t \mathrm{d}t + \sigma S_t \mathrm{d}W_t $$
In this equation, $S_t$ is the random viable in $t$ point, $\mu$ is the average increase rate in unit time, and the $\sigma$ is the volatility rate of random viable, and the $W_t$ is the basic Brownian Motion, which has $\mu$ with 1 and $\sigma$ with $t$.
We could solve this equation with the method of solving stochastic differential equations. so we could have the general answer of this SDE like this. $$ S_t = S_0 + \int_0^t \mu S_i \mathrm{d}i + \int_0^t \sigma S_i \mathrm{d}W_i $$
According to the Ito’s lemma, we could solve this equation with this form. $$ \frac{\mathrm{d}S_t}{S_t} = \mu \mathrm{d}t + \sigma \mathrm{d}W_t $$
Then we integrate the equation on both sides. $$ \ln S_t  \ln S_0 = (\mu  \frac{1}{2}\sigma^2)t = \sigma W_t $$
We could time the $e^x$ on both side then the formula would be like this. $$ S_t = S_0 \exp [(\mu  \frac{1}{2}\sigma^2)t + \sigma W_t] $$
Now we have solved this SDE and the GBM could be stimulated, and the Probability Density Function would be like this. $$ p(S) = \frac{1}{S \sigma \sqrt{2\pi t}}\exp [\frac{[\ln(\frac{S}{S_0})(\mu\frac{\sigma^2}{2})t]^2}{2\sigma^2t}] $$
So what we need to do is like this.
If we know this process well, we would get the program easily, first of all we need to get $\mu$ and $\sigma$.
We could get these number by the code below.


After running this code we could get these two parameters and the next thing we should do is construct the GBM and use Monte Carlo to give the much path.
We could achieve this goal by this code.


Then we could get the histogram like this.
And the stimulated path would there and we could get all path on the figure.
I have stimulated this GBM for 10000 times and actually it is up to you.
Then we could use mean method to get the prediction or any other method.
At last, I think this method is a classic way which combines the physic and finance, if the physical theory would be used in financial market analysis, it could be so usefull.
]]>As I said in last note, there are so many chaotic series in financial market, like the stock price or other asset prices, most of these series don’t have the enough features of custom time series analysis, this kind of prices is usually influenced by many factors, so the series always see a irregular trend, some series look like a random walk such as stock prices. In my mind, SDE is also a good tool of analyzing the stock price, I’ll tell this method in the later notes.
Because of the irregular features, the phase space reconstruction becomes, it make the chaotic series could be reflected in the high dimension, and the typical example is the Lorenz curve, it is regarded as a great model in weather prediction and weather dynamic system, but I think it’ll also be a good tool in financial field.
Phase Space Reconstruction is reflected method in time series, it could make the series reconstruct in the many other dimonsions by delay and embedding. Then I’ll talk about this method gradually.
Takens Embedding was born in 1981, it said the chaotic series like this form could be reconstructed in the space. $$ y(i) = (x(i),\dots,x(i+(d1)\tau)), \quad where 1 \leq i \leq n(d1)\tau $$
In this formula, the d means the dimension of vector $y(i)$, it is usually called embedding dimension and $\tau$ is the delay time.
As we can see, the new generate data is depend on the embedding dimension $d$ and the delay time $\tau$, but I think the most important thing for beginner is that the new data absolutely comes from the former data, that means every new data in the new space is one of the old data!
So the keys of this method is how to calculate the numbers of embedding dimension and delay time.
The first method of calculate delay time is ACF, and we need to find the minimum number in this process and when the $\rho$ is close to the $1e^{1}$, then we could choose the $x$ as the lag of the delay time. $$ R(\tau) = \frac{1}{n}\sum_{i=1}^{n\tau}x(i) \times x(i+\tau) $$
But the problem of this is the method is the ACF shows the linear relationship in the delay time, but we has defined the system is nonlinear, so we need the other method to choose the lag.
The other method is AMI, which is a method to calculate the information in each series. If we have a series $X$, and the $Y$ is a delay series. with $N\tau$ observations, where $\tau$indicates the delay; the average mutual information $I_{Y;X}$ between both time series can be expressed, in probabilistic terms. $$ I_{Y;X} = I_\tau = \sum_{i=1}^{N_c}\sum_{j=1}^{N_c} P(x(i), y(j)) \mathop{log}\frac{P(x(i),y(j))}{P(x(i))P(y(j))} \\ \quad \quad \quad = \sum_{i=1}^{N_r}P(s_i,s_{i+\tau})\mathop{log}\frac{P(s_i,s_{i+\tau})}{P(s_i)P(s_{i+\tau})} $$
where $N_c$ is the number of cells containing points, with nonzero probability, and $N_r$ is the number of routes in the state space.
$$ I_{Y;X} = H_Y + H_X  H_{X,Y} $$
And we could program this method easily by MATLAB and we could get the picture as this.
Then we need to determine the embedding dimensions. In this period, we could use the FNN(False Nearest Neighbors) method to mesure the fittest number.
Actually, we reconstruct the data is reflect the data to the other dimension, in other words, we need to extend the track of the single series and expand this data to a different space.
We defined the every point in the series has a Euclidean distance with other points. $$ R_i(d) = y_i(d)  y_{n(i,a)}(d)_2 $$
If the dimension become $d+1$ frome $d$, the distance would be $R_i(d+1)$, like this. $$ R_i(d+1)^2 = R_i(d) + x(i+d\tau)x(n(i,d)+d\tau)_2^2 $$
If the $R_i(d+1)$ is much higher than $R_i(d)$, we could think this closed point is wrong. $$ a_1(i,d) = \frac{x(i+d\tau)x(n(i,d)+d\tau)_2}{R_d(i)} $$
When the $a_1(i,d) \in [10,50]$, we could think the point is wrong.
Then we could use MATLAB again and the picture could be showed as below.
Then we could get the new space and become the new space from the old data.
After we finish this process, we could do the prediction of the nonlinear system.
The data we should understand is the $n(d1)\tau$ vector like this.
$$ \vec{y}_1 = y_1(d) = (x(1),\dots, x(1+(d1)\tau)) \\ \vec{y}_2 = y_2(d) = (x(2),\dots, x(2+(d1)\tau)) \\ \dots \\ \vec{y}_i = y_i(d) = (x(i),\dots, x(i+(d1)\tau)) \\ \dots \\ \vec{y}_{n(d1)\tau} = y_{n(d1)\tau}(d) = (x(n(d1)\tau),\dots, x(n)) $$
Then we could set a dynamic system in Euclidean space with $d$ dimension, and we could predict this system by many methods like machine learning, deep learning or least square. $$ \vec{y}_{i+1} = F(\vec{y_i}) $$ The connected function $F(\vec{y_i})$ would be easy to estimate.
Finally, I think this way of treating the data could keep the compeleted information and the new space could be much fitter to do the analysis.
]]>In the field of my study, the method of times series forecasting is so similar, almost every method is based on the trend and seasonal, but it is clear that not all times series follow these rules. Some times series come from the real volatility, and it doesn’t have the obvious features of trend and seasonal. In specific, this kind of series is usually defined as chaos series according to the chaos theory.
But how to forecast this kind of series? The jerk equation is the simplest method to do this by chaos theory.
The jerk equation defined the three basic concepts.
First concept is velocity, it’s the rate of change in time of the price, it could be showed in this equation.
$$ v = \frac{\mathrm{d}x}{\mathrm{d}t} $$
The second concept is the acceleration, it means the rate of change of velocity, or the second derivative of the price. $$ a = \frac{\mathrm{d}v}{\mathrm{d}t} = \frac{\mathrm{d}^2 x}{\mathrm{d}t^2} $$
Then the third concept and the most important concept is Jerk, it means the rate of change of acceleration, or the third derivative of the price.
$$ j = \frac{\mathrm{d}a}{\mathrm{d}t} = \frac{\mathrm{d}^2 v}{\mathrm{d}t^2} = \frac{\mathrm{d}^3 x}{\mathrm{d}t^3} $$
After the $v$, $a$, $j$, are defined by the price. The differential equations form could be showed as below. $$ J(x^{\prime \prime \prime},x^{\prime \prime},x^{\prime},x) = 0 $$
This equation is called jerk equations, this kind of equation is the minimal setting for solutions showing chaotic behavior. Thus,the approach to model price movement will be model its behavior with the equation of the form. $$ A\frac{\mathrm{d}^3 x}{\mathrm{d}t^3} + B\frac{\mathrm{d}^2 x}{\mathrm{d}t^2} + C\frac{\mathrm{d} x}{\mathrm{d}t} + Dx +E = 0 $$
We could convert this equation to the form below.
$$ A x^{\prime \prime \prime} + B x^{\prime \prime} + C x^{\prime} + D x +E =0 $$ In this equation, the A, B, C, D and E are the real numbers and is the parameters we need to estimate.
When E = 0, this differential equation could be simplified as below. $$ A\lambda^3 + B\lambda^2 + C\lambda + D = 0 $$
To solve this equation, we could get the $x(t)$ as this. $$ x(t) = C_1 e^{\lambda_1 t} + C_2 e^{\lambda_2 t} + C_3 e^{\lambda_3 t} $$
The jerk equation could become this form. $$ x_0 = C_1 +C_2+ C_3 \\ v_0 = C_1\lambda_1 + C_2\lambda_2 + C_3\lambda_3 \\ a_0 = C_1\lambda_1^2 + C_2\lambda_2^2 + C_3\lambda_3^2 \\ j_0 = C_1\lambda_1^3 + C_2\lambda_2^3 + C_3\lambda_3^3 $$
To give the time change, we do the discretization of this equation. $$ x^{\prime} \approx \frac{x_{i+1}x_i}{\Delta t} $$
$$ x^{\prime \prime} \approx \frac{x_{i+1}2x_i+x_{i1}}{\Delta t^2} $$
$$ x^{\prime \prime \prime} \approx \frac{x_{i+1} 3x_i+3x_{i1}x_{i1}}{\Delta t^3} $$
Then we could regard this system as a linear system and solve this equations by a leastsquares method.
$$ A\vec{x} = \vec{b} $$
Then we could get the estimate parameters easily. After doing the estimation, we should consider the predictor method of these equations.
We could convert the jerk equation to this form.
$$ A(x_{i+1}3x_i+3x_{i1}x_{i2})+B(x_{i+1}2x_i+x_{i1})+C(x_{i+1}x_i)+Dx_i+E = 0 $$
The price next day could be approximated by following equation, with the linear system.
$$ x_{i+1} = \frac{E+Ax_{i2}x_{i1}(3A+B)x_i(3A2BC+D)}{A+B+C} $$
At last, we could push the days by this equation and try to do the prediction through this equation. It’s a good method of doing prediction.
]]>Today, I’ll talk about the method of estimating the ActionValue Functions in Reinforcement Learning which usually named $Q(s,a)$.
Actually, this kind of $Q(s,a)$ has two types, they were defined as below.
Actionvalue function for policy $\pi$. $$ Q_{\pi}(s_t,a_t) = \mathbb{E}[U_tS_t = s_t, A_t = a_t] $$
It means the each action’s value of each agent’s each state, it can be showed as this.
$a_1$  $a_2$  $a_3$  $a_4$  

$s_1$  1  2  3  4 
$s_2$  2  1  0  5 
$s_3$  3  6  8  9 
And the other type os the optimal actionvalue function.
$$ Q^*(s_t,a_t) = \mathop{max}\limits_{\pi}Q_{\pi}(s_t,a_t) $$
As we can see, this function means the highest value in $Q_{\pi}(s_t,a_t)$, it’s easy to understand how an agent do the decision between these actions depend on the the state it has and the ecvironment.
To understand this thing, we define a situation.
Goal: Win th game(maximize the total reward).
Question: If we know $Q^*(s,a)$, what is the best action?
$Q^*$ is and indication for how good it is for an agent to pick action a while being in state s. Then this agent will choose the best action and try its best to achieve the goal.
In my understand, it is a simulation of a person to do decision, but the computer could calculate the expected value of the whole process and do the best choice, and every state do the best action in the every environment.
But the challenge is: How we know the $Q^*(s,a)$?
The general solution is Deep Q Network(DQN).
Like the Neural Network $Q(s,a; w)$, we could use it to approximate $Q^*(s,a)$.
For example, we let the computer to estimate the distance between Hangzhou and Beijing, and the computer will give the estimate: $Q(w)$ = 1000 minutes, but after the the agent achieve the real goal for the first time, it just took the 900 minutes, it is clear that the 900 minutes is more reliable estimate than 1000.
So the Loss in this situation is. $$ Loss:L = \frac{1}{2}(Q(w)y)^2 $$
The Gradient algorithm is. $$ \frac{\partial L}{\partial w}=(1000900) \times \frac{\partial Q(w)}{\partial w} $$
As we can see the $(1000900)$ is the TD error.
Then we could make it more general.
In the same situation, but this time when we arrived at Hefei, our car don’t work, so we have to stop our travel, but this time we get the part real minutes between Hangzhou and Hefei and computer would give the estimate of the left way. The computer could also estimate the minutes between Hangzhou and Hefei at start, such as 400, but the real minutes is 300. So the error is 100.
Then we could give this general equation.
$$ T_{Hangzhou \rightarrow Beijing} \approx T_{Hangzhou \rightarrow Hefei} + T_{Hefei \rightarrow Beijing} $$
so the calculate equation is . $$ Q(s_t,a_t;w) \approx r_t + \gamma \times Q(s_{t+1},a_{t+1};w) $$
The $r_t$ means the real data in this travel. Then we could get the return function easily. $$ U_t = R_t + \gamma \times R_{t+1} + \gamma^2 \times R_{t+2} + \gamma^3 \times R_{t+3} + \dots $$
We could put the $\gamma$ out, the equation change like this. $$ U_t = R_t + \gamma \times (R_{t+1} + \gamma \times R_{t+2} + \gamma^2 \times R_{t+3} + \dots) $$
Then we could get the identity: $ U_t = R_t + \gamma \times U_{t+1}$
To get the $Q(s_t,a_t;w)$, it means the estimation of $\mathbb{E}[U_t]$, the $Q(s_{t+1},a_{t+1};w)$, means the estimation of $\mathbb{E}[U_{t+1}]$, So the method equation is.
$$ Q(s_t,a_t;w) \approx \mathbb{E}[R_t+\gamma \times Q(S_{t+1},A_{t+1};w)] $$
The left of this equation is the prediction and the right of this equation is the TD target. SO we could get this equations.
Prediction:$Q(s_t,a_t;w_t)$
TD target:
$$ y_t = r_t + \gamma \times Q(s_{t+1},a_{t+1};w_t) \\ \quad \quad = r_t + \gamma \times \mathop{max}\limits_{a}Q(s_{t+1},a;w_t) $$
Loss:$L_t=\frac{1}{2}[Q(s_t,a_t;w)y_t]^2$
The ultimate goal: $$ Q^*(s_t,a_t) = \mathop{max}\limits_{\pi}\mathbb{E}[U_tS_t = s_t, A_t = a_t] $$
The $Q(s,a;w)$ is a neural network parameterized by $w$, we could input the observed state $s$ then it’ll output the scores for every action $a \in \mathop{A}$.
The TD target is similar to the famous equation named Bellman Euqation, we could remember this equation because it’s the heart of the reinforcement learning.
Bell equation: $$ Q_{*}(s_t,a_t) = \mathbb{E}_{S_{t+1}~p(·s_t,a_t)}[R_t+\gamma \times \mathop{max}\limits_{A \in \mathop{A}}Q(S_{t+1},A)S_t = s_t,A_t=a_t] $$
At last, the method based on value could solve the problem like this and I considered if this estimate could be used in the evaluate question, I’m always thinking about this, if someone has the same thought, please tell me in the comment module.
]]>At the beginning of whole analysis, you need to finish installing the MATLAB(you could see a lot of tutorial on website whatever it’s Cracked or Genuine) and know some basic programming language.
Then you need to download the MATLAB toolbox of Dynamic Copula Toolbox 3.0, you could click here, and then put it into your workspace.
First of all, you need to import your data from excel or other files, and named the variable like data or any other names, then remember it.
Secondly, you should click twice on the Dynamic Copula Toolbox folder and enter this, then you could start your analysis.
Then you could enter the command into the terminal of MATLAB, following with my step.
Now, you could enter this command into the terminal below.


Then you’ll see the window like this.
Choose “Garch model for each series”, and it will start to estimate the Garch model for marginal distribution.
Then you would see the output like this.
If these series have autocorrelation, you should enter “1”, but if they don’t, just enter “0” is enough.
Then press the keyboard “enter”, it’ll continue, and the window like this will appear.
You should choose a model for what you want, and click it. Then it’ll go to the next period.
If you see this, you’ll fell a bit close. It’s the distribution of residuals, you just need to choose the fittest one. Even though the Skew T and T is the most general. Choose it and click it!
Then you’ll see this window, the IFM means TwoStage Maximum Likelihood Estimation, I think it is more general, and you’ll see the configuration in the command window.
Then you need to input the next command.


After running, you could see a window appear.
As for me, it’s really enough to click the “input defaults”, then the terminal will start running and the window below will become a few seconds later.
Generally, you could click yes for the estimation of standard errors.
Then the estimate result would be there.
You just need to press enter then it’ll be going on. The second same window will appear soon, you just need to click “yes” again and the estimation is done.
Of course, it’s a good chance for you to save the variable and you should input the code below to do this.


Now the variable varargout is the standard deviation probability residual integral transforms the sequence that follows for Garch model.
You’d like to start the copula model with this variable.
Just input this code into the terminal.


It’ll start estimate for Copula Model and you’ll see the same window again.
But at this time, you should click “Copula” and the window below would happen.
You could choose the copula family you want, just click it and you will see another window.
You can decide which style you need, the “static” means stable type, and the “DCC” means the dynamic type.
After clicking, you will see the configuration of copula estimation on command window, then you need to input the code below and it’s the last step.


You’ll see it again and just choose “input defaults”.
Then it has started to estimate and you can choose if you need to calculate the standard errors and click “yes”, the whole process is completed.
You can calculate the $VaR$, $CoVaR$, $\Delta CoVar$ at this time, here are some commands to help you calculate them.


The dynamic correlation coefficient is in the Rt of evalmodel and the parameters variable includes the copula parameters that it estimated just now.
You could also to draw it like the figure below.
Lastly, you could calculate the $\Delta CoVaR$ by MATLAB easily and the code I provided below might help you.


The picture always looks like this.
At last, I don’t know why I didn’t find any usefull information about this toolbox on the website and in some places, even need to spend money on these code, so I think my work would help some people who also feel annoyed like me when they are researching the dynamic copula model.
]]>To avoid the compelax searching process, I decide to note this easy method to finish it.
Firstly, you need set the env
parameter and make it as below.


Then you should edit the head of the config.yml
.


Lastly, you just add the command under the ../theme/layouts/_default/partials/head.html


Then you should run the code below on the teiminal to produce the “public” folder.


Then you just deploy it on the server again and wait a few minutes and it’ll appear on the Google Analytics website.
]]>Yesterday I was deploying this website to my server on Oracle Cloud, but I tried so many times of applying the SSL permission with BTPanel and the server broke down every time. It toke tons of time and annoyed me.
Even though the BTPanel is an excellent tool, but it seems that it couldn’t work when it meets Oracle Cloud. So after a few tries, I decided to find a new panel to deal with this problem.
After my searches, I find a panel developed by Chinese group named mdserverweb panel, which is so similar to the BTPanel.
The developer said that this panel is based on BTPanel and just do a little change.
To set a website with this panel, Firstly we need to install this panel. With different systems, enter this command to the terminal and wait a few minutes about 12m or so.


This command will match your VPS’s system automatically and what you need to do is just waiting!
After installing, don’t forget to open the ports I had written on the last note.(Like the BTPanel)
The only difference between this panel and BTPanel is that the plugins, this panel use the plugins much smaller like OpenResty not Nginx, but it has the enough abilities to construct our own websites.
In this example, I find the mdserverweb has a better experience then BTPanel, and I would like to choose this panel more in the future
]]>As we need to construst a website belonging to us, BTPanel is a convenient tool for beginner.
Centos


Ubuntu/Deepin


Debian


Fedora




After we run these codes, we just need to wait and the informations like address, username and passward will appear on the terminal.
But after we input the address on the browsers, we would find that the website can’t work!(Even we change the security list on Oracle Cloud!) The reason of this problem is the port of the cloud server didn’t open and we need to run the code below, then this problem will be solved.


The we enter the address again, and we could see the correct result and the BTPanel will work on the browser!
]]>Recently, I find I always forget the way of connecting the SSH with the file named xxxx.key and xxxx.key.pub, these files come from the Oracle Cloud VPS and it’s the only method to connect it.
So I start to record this a bit compelax method of connection.
First of all, we need change the permission of the file named xxxxx.key because if we don’t do this, it will get fault when we try connecting it, it would say “The permission 640 is too open …”, so we need to convert the 640 to 600(only read).


and then we just cd
the dictionary of the file existed and run the code below:


Then we could connect it successfully.(the username is the cloud server username, for example my system is Ubuntu 20.04.6 on Oracle Cloud, so my username is ubuntu)
]]>As a beginner, the definations are so importnant for us. Only we are clear about these compelax things, then we could be much easier to understand the knowledge later.
Note: The lower case letters means the observation variable, the uppercase letters mean the random variable.
Return: $$ U_t = R_t+\gamma R_{t+1}+\gamma^2 R_{t+2}+ \dots $$
Actionvalue function: $$ Q_\pi(s_t,a_t) = \mathbb{E}[U_ts_t,a_t] $$
Optimal actionvalue function: $$ Q^*(s_t,a_t) = \mathop{max}\limits_{\pi}Q_\pi(s_t,a_t) $$
Statevalue funtion:
$$ V_\pi(s_t) = \mathbb{E}_{A}[Q_{\pi}(s_t,A)] $$
During the iteration process, the agent can be controlled by either $\pi(as)$ or $Q^*(s,a)$.
So these two things are the targets we should estimate and we will learn some methods to finish this process in the later lessions.
]]>