-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathatom.xml
421 lines (225 loc) · 341 KB
/
atom.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>catinsides.github.io</title>
<link href="https://catinsides.github.io/atom.xml" rel="self"/>
<link href="https://catinsides.github.io/"/>
<updated>2025-02-12T08:43:14.290Z</updated>
<id>https://catinsides.github.io/</id>
<author>
<name>catinsides</name>
</author>
<generator uri="https://hexo.io/">Hexo</generator>
<entry>
<title>记一次VOIP服务端开发并对接AI</title>
<link href="https://catinsides.github.io/2025/02/12/%E8%AE%B0%E4%B8%80%E6%AC%A1VOIP%E6%9C%8D%E5%8A%A1%E7%AB%AF%E5%BC%80%E5%8F%91%E5%B9%B6%E5%AF%B9%E6%8E%A5AI/"/>
<id>https://catinsides.github.io/2025/02/12/%E8%AE%B0%E4%B8%80%E6%AC%A1VOIP%E6%9C%8D%E5%8A%A1%E7%AB%AF%E5%BC%80%E5%8F%91%E5%B9%B6%E5%AF%B9%E6%8E%A5AI/</id>
<published>2025-02-12T07:47:42.000Z</published>
<updated>2025-02-12T08:43:14.290Z</updated>
<content type="html"><![CDATA[<p>最近接到了一个新需求，是通过 WebRTC 通话对接到一个 AI 语音助手。 </p><p>现在正是 AI 的大热潮，做这样一个功能，也是个好机会，可以学习一下相关知识。 </p><p>对接的 AI 使用的是 <a href="https://www.ultravox.ai/">Ultravox.ai</a>。 </p><p>虽然他们提供了一些集成到电话的功能，但是都是些国外的服务商，而我们需要集成到自己的系统上，所以不满足需求。 </p><p>首先要解决的问题是，在客户端与服务端实现完整的通话流程。我不太理解的是，为什么不使用 Asterisk 或者 Freeswitch，而是要自己造轮子。 </p><p>在我接手之前，有同事已经做了一个 nodejs 版的 demo 中的 demo，实现了 SIP 和 SDP 消息的对接，媒体使用 FFmpeg 处理，总体流程是 OK 的。 </p><p>但是由于是 demo 级别的，没有工程化处理，也没有考虑后续的扩展性，再往上堆彻功能的话有些吃力。尽管如此我还是在他的 demo 之上，完成了 FFmpeg 转码和 Azure 语音转文字等功能。 </p><p>为了学习如何在服务端更好的处理会话，寻找更好的编程模式，我开始到 github 寻找其他开源项目学习。一开始只找 nodejs 相关的项目，找到了 node.js-sip 这个项目。 </p><p>这个项目整体结构比先前的 demo 稍好一些，但是也很久不维护了，代码看起来也很随意，运行起来遇到不少问题。自己 fork 过来后，改了一段时间，还是受不了，继续寻找其他的项目。 </p><p>在一番搜索下，发现 nodejs 相关的库实在是太少了。因为我自己之前自学了 golang，感觉用它写项目也能 hold 住，就开始找相关的库。 </p><p>还真别说，找到了质量很高的库 diago。基本上可以说能够开箱即用，可以直接对接到我们系统的 WebRTC 通话，代码很优雅，功能丰富。 </p><p>在使用 diago 开发的过程中，还遇到了不满足自身需求的情况，我尝试修改了源码提了 PR（人生第一次给外国人提），这给我激动的，生怕给我拒绝了。 </p><p>过了几天一看，作者提了一些修改建议，我修改之后作者就给合并了，给我高兴发财了嘿嘿嘿。 </p><p>话说回来，在实现与 WEBRTC 对接之后，开始考虑如何与 Ultravox 对接。 </p><p>Ultravox 提供的方案是，使用 http 请求创建通话，然后返回一个 websocket 连接。客户端的媒体数据从这个 ws 连接发给 ultravox，ultravox 再从这个 ws 返回媒体数据，我再将媒体数据写回客户端。 </p><p>媒体的流程大概是这样的 </p><p>RTP -> websocket -> RTP</p><p>从 WebRTC 过来的媒体数据要发送到 websocket 涉及到格式转换，diago 方便地提供了这些方法，可以将 g711 格式转换成 PCM，这样就可以直接写入到 websocket 中去了。 </p><p>ultravox 返回的是 PCM 数据，同样可以使用 diago 提供的方法再转换为 g711，这样就节省了很多力气。 </p><p>ultravox 除了返回媒体数据外，也返回了数据消息，如 AI 语音转文字的结果，后续可以将这些文字结果入库，根据业务需求调整。 </p><p>现在回想起整个的开发过程，虽然描述起来感觉很轻松，但是期间也遇到了不少困难。期间也通过了不少 GPT 解决了问题，现在的 AI 用得好的话，也是能武装我们这些开发人员的。</p><p>不知道怎么接着往下写了，先这样吧~</p><p>——————<br>本文为<a href="https://github.com/Catinsides">个人原创</a>，转载请署名且注明出处。 </p>]]></content>
<summary type="html"><p>最近接到了一个新需求，是通过 WebRTC 通话对接到一个 AI 语音助手。 </p>
<p>现在正是 AI 的大热潮，做这样一个功能，也是个好机会，可以学习一下相关知识。 </p></summary>
<category term="笔记" scheme="https://catinsides.github.io/tags/%E7%AC%94%E8%AE%B0/"/>
<category term="AI" scheme="https://catinsides.github.io/tags/AI/"/>
<category term="VOIP" scheme="https://catinsides.github.io/tags/VOIP/"/>
</entry>
<entry>
<title>记一次浏览器插件开发经历</title>
<link href="https://catinsides.github.io/2024/08/07/%E8%AE%B0%E4%B8%80%E6%AC%A1%E6%B5%8F%E8%A7%88%E5%99%A8%E6%8F%92%E4%BB%B6%E5%BC%80%E5%8F%91%E7%BB%8F%E5%8E%86/"/>
<id>https://catinsides.github.io/2024/08/07/%E8%AE%B0%E4%B8%80%E6%AC%A1%E6%B5%8F%E8%A7%88%E5%99%A8%E6%8F%92%E4%BB%B6%E5%BC%80%E5%8F%91%E7%BB%8F%E5%8E%86/</id>
<published>2024-08-07T09:00:00.000Z</published>
<updated>2025-02-12T08:43:07.543Z</updated>
<content type="html"><![CDATA[<p>最近做了一个浏览器插件的项目，私以为还是比较有意思的，在此记录一下经历。 </p><p>这个浏览器插件的主要功能是，提供一个拨号盘界面，可以进行呼入、呼出的功能。再额外添加一些围绕着呼叫的辅助功能，比如：联络记录，通讯录，弹屏等等。 </p><h2 id="WXT-介绍"><a href="#WXT-介绍" class="headerlink" title="WXT 介绍"></a>WXT 介绍</h2><p>为了提高开发效率，我直接查找了可以用的插件开发框架，经过一番搜索最终选择了 <a href="https://wxt.dev/">WXT</a>。</p><p>它可以使用流行的前端框架进行开发，它是建立在 <code>Nuxt</code>之上的，而我对 <code>Vue</code> 还算比较熟悉，也就选择了 <code>Vue</code> 这条技术路线一路到底。 </p><p>即使现在看来，整个开发过程的体验还是很轻松加愉快的。不足之处是，说明文档不是很全，有很多待补充内容，如果找不到想要的内容，只能到 issues 中寻找。 </p><p>打开 <code>WXT</code> 的官方网站，找到 <code>Get Stared</code>，初始化项目, </p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">pnpm dlx wxt@latest init <project-name></span><br></pre></td></tr></table></figure><p>命令执行完成之后，会生成一些模板代码，具体有什么用可以到官网查看 <a href="https://wxt.dev/guide/directory-structure">Link</a> 。 </p><p><code>WXT</code> 遵循的是 <code>约定优于配置</code>，每个目录内的文件在最终编译时会转换为对应的文件。 </p><p>例如：<code>components</code> 内的文件会转换为组件，<code>background</code> 内的文件会转为 <code>ServiceWorker</code>，<code>*.content.ts</code> 会转为 <code>content scripts</code>。</p><p>如果熟悉 <code>Nuxt</code> 的话，<code>composables</code>, <code>hooks</code>, <code>utils</code> 内的文件可以自动导入。像是在使用全局的函数，用起来很方便。</p><h2 id="插件内的概念"><a href="#插件内的概念" class="headerlink" title="插件内的概念"></a>插件内的概念</h2><p>在开始动手之前，需要先了解一下浏览器插件的概念，有助于开发。</p><h3 id="popup"><a href="#popup" class="headerlink" title="popup"></a>popup</h3><p><code>popup</code> 是用户点击插件图标时，弹出的窗口。这个窗口有大小限制，为 800x600。 </p><p>这个窗口内的内容，可以理解为就是html。框架内编写的vue组件是在这里渲染的。对应 <code>WXT</code> 的 <code>entrypoints</code> 目录中的 vue 文件。 </p><p>每次打开 <code>popup</code> 时，vue代码都会重新加载。如果想实现一些状态共享的功能，在 <code>popup</code> 关闭时，状态都会清空，所以需要存到 <code>storage</code> （后面讲）中在打开时再次读取。</p><h3 id="background-scripts"><a href="#background-scripts" class="headerlink" title="background scripts"></a>background scripts</h3><p><code>background scripts</code> 是在浏览器后台运行的脚本。在 manifest v3 中，backgound scripts 就是 service worker. </p><p>对应 <code>WXT</code> 的 <code>entrypoints/background</code> 目录中的文件。</p><p>background scripts 不受跨域影响，可以监听来自 <code>popup</code> 和 <code>content scripts</code> 的消息。</p><h3 id="content-scripts"><a href="#content-scripts" class="headerlink" title="content scripts"></a>content scripts</h3><p><code>content scripts</code> 是在浏览器页面中运行的脚本。在页面加载后会注入到页面中。可以操作 DOM 元素。</p><h3 id="storage"><a href="#storage" class="headerlink" title="storage"></a>storage</h3><p><code>storage</code> 用来存储数据。类似 <code>localStorage</code>，使用时必须加上前缀，<code>local:</code>、<code>session:</code>、<code>sync:</code> 和 <code>managed:</code>。 </p><p>与 <code>localStorage</code> 不同，<code>storage</code> 可以监听值的变化，也可以存储对象值。 </p><p><code>storage</code> 的数据可以在 <code>popup</code>，<code>background scripts</code> 和 <code>content scripts</code> 之间共享。 </p><p>我把它当作 <code>Redis</code> 来用，暂时没发现姿势有什么不对劲。</p><h3 id="message"><a href="#message" class="headerlink" title="message"></a>message</h3><p><code>message</code> 用来在 <code>popup</code>，<code>background scripts</code> 和 <code>content scripts</code> 之间传递消息。 </p><p>主要用法为</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// 接受消息</span></span><br><span class="line">browser.<span class="property">runtime</span>.<span class="property">onMessage</span>.<span class="title function_">addListener</span>(<span class="function">(<span class="params">message, sender, sendResponse</span>) =></span> {</span><br><span class="line"> <span class="comment">// do something</span></span><br><span class="line"> <span class="title function_">sendResponse</span>();</span><br><span class="line"> <span class="comment">// 如果发送端需要回复 要返回return true 表明是一个异步消息</span></span><br><span class="line"> <span class="keyword">return</span> <span class="literal">true</span>;</span><br><span class="line">});</span><br><span class="line"><span class="comment">// 发送消息</span></span><br><span class="line">browser.<span class="property">runtime</span>.<span class="title function_">sendMessage</span>({<span class="attr">message</span>: <span class="string">'hello'</span>});</span><br></pre></td></tr></table></figure><h2 id="插件功能介绍"><a href="#插件功能介绍" class="headerlink" title="插件功能介绍"></a>插件功能介绍</h2><p>插件通话功能是由 <code>jssip</code> 实现的。页面上实现了登录注册，指定好后端地址，就可以使用通话功能了。 </p><p>插件的页面组件使用了 <code>vant</code>.</p><h3 id="拨号盘和话务功能"><a href="#拨号盘和话务功能" class="headerlink" title="拨号盘和话务功能"></a>拨号盘和话务功能</h3><p>这部分我使用 vue 画了一个拨号盘的界面用于用户交互。话务功能 <code>jssip</code> 中有现成的函数，可以拨号，挂断，静音等，直接调用相关函数就可以了。 </p><p>拨号盘界面需要注意的是，要根据不同的话机状态，显示不同的样式。 </p><p>因为 <code>jssip</code> 要获取用户媒体和创建 Audio 用于播放铃声，这部分代码只能放到 <code>content scripts</code> 脚本中。 </p><p>个人暂时觉得还是不太完善，在网上查资料可以有 <code>offscreen.html</code> 和 <code>background.html</code> 的其他方法，暂未成功。 </p><p><code>popup</code> 中的状态，在用户关闭后就会丢失，而且这个窗口很容易被关闭。每次打开时都需要从 <code>content scripts</code> 中同步状态，这也就用到了 <code>storage</code> 和 <code>message</code>。 </p><p>来电通知使用了 <code>notifications</code>，如下代码。这一部分倒是没遇到什么问题，<code>jssip</code> 收到来电后，使用 <code>message</code> 发到 <code>background scripts</code> 就可以了。</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">browser.<span class="property">notifications</span>.<span class="title function_">create</span>({</span><br><span class="line"> <span class="attr">type</span>: <span class="string">''</span>,</span><br><span class="line"> <span class="attr">title</span>: <span class="string">''</span>,</span><br><span class="line"> <span class="attr">iconUrl</span>: <span class="string">''</span>,</span><br><span class="line"> <span class="attr">message</span>: <span class="string">''</span>,</span><br><span class="line">});</span><br></pre></td></tr></table></figure><p>为了加强插件状态的直观性，在通话和来电时改变图标，可以使用</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">browser.<span class="property">action</span>.<span class="title function_">setIcon</span>({});</span><br></pre></td></tr></table></figure><p>除了 <code>jssip</code> 的 <code>websocket</code> 外，插件还需要一个新的 <code>websocket</code> 获取后台系统的消息，以实现弹屏功能。这一部分也放到了 <code>content scripts</code> 脚本中，暂时没遇到什么坑，轻松加愉快。</p><h3 id="通讯录"><a href="#通讯录" class="headerlink" title="通讯录"></a>通讯录</h3><p>通讯录可以直接使用 <code>vant</code> 内的组件和 <code>storage</code> 来实现。 </p><p>在使用 <code>storage</code> 存储的时候需要注意，数据是插件内全部可见的。也就是即使我这个插件需要不同的用户先登录后再使用，如果用户A保存了通讯录，用户B登录后也能访问到。 </p><p>所以在保存这些数据时，需要给数据打上标记，让用户只能访问自己建立的数据。</p><h3 id="弹屏功能"><a href="#弹屏功能" class="headerlink" title="弹屏功能"></a>弹屏功能</h3><p>弹屏功能是收到来电后，可以打开指定的网页内容。网页可以由用户自行填写，支持内容替换。这样在和其他CRM系统集成的时候，可以方便的调用第三方系统内的功能。 </p><p>这一部分也比较简单，使用 <code>message</code> 然后在 <code>background scripts</code> 脚本中处理。 </p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">chrome.<span class="property">tabs</span>.<span class="title function_">create</span>({ url });</span><br><span class="line"><span class="comment">// or</span></span><br><span class="line">chrome.<span class="property">tabs</span>.<span class="title function_">update</span>({ url });</span><br></pre></td></tr></table></figure><h3 id="国际化"><a href="#国际化" class="headerlink" title="国际化"></a>国际化</h3><p>这一部分使用了 <a href="https://vue-i18n.intlify.dev/">vueI18n</a> </p><p>可以参照 <code>WXT</code> 的<a href="https://github.com/wxt-dev/wxt-examples/tree/main/examples/vue-i18n">示例</a> </p><p>没有遇到什么坑点，唯一要注意的是，在 <code>background scripts</code> 中，要这么用 </p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">i18n.<span class="property">global</span>.<span class="title function_">t</span>(<span class="string">''</span>, {}, { <span class="attr">locale</span>: <span class="string">'en'</span> });</span><br></pre></td></tr></table></figure><h2 id="界面截图"><a href="#界面截图" class="headerlink" title="界面截图"></a>界面截图</h2><p float="left"> <img alt="screenshot-1" src="https://github.com/Catinsides/catinsides.github.io/blob/source/assets/dialpad-screenshots-1.png?raw=true" width="200"> <img alt="screenshot-2" src="https://github.com/Catinsides/catinsides.github.io/blob/source/assets/dialpad-screenshots-2.png?raw=true" width="200"> <img alt="screenshot-3" src="https://github.com/Catinsides/catinsides.github.io/blob/source/assets/dialpad-screenshots-3.png?raw=true" width="200"> <img alt="screenshot-4" src="https://github.com/Catinsides/catinsides.github.io/blob/source/assets/dialpad-screenshots-4.png?raw=true" width="200"></p><h2 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h2><p>经过这个项目，我熟悉了浏览器插件的开发流程和概念。 </p><p>幸好用了框架，没有让代码乱成一锅粥。 </p><p><code>vue</code> 生态中的资源也能很好的利用上。我想如果后续迭代的话，也很方便快捷。</p><p>——————<br>本文为<a href="https://github.com/Catinsides">个人原创</a>，转载请署名且注明出处。 </p>]]></content>
<summary type="html"><p>最近做了一个浏览器插件的项目，私以为还是比较有意思的，在此记录一下经历。 </p>
<p>这个浏览器插件的主要功能是，提供一个拨号盘界面，可以进行呼入、呼出的功能。再额外添加一些围绕着呼叫的辅助功能，比如：联络记录，通讯录，弹屏等等。 </p></summary>
<category term="Javascript" scheme="https://catinsides.github.io/tags/Javascript/"/>
</entry>
<entry>
<title>记一次录音转写项目经历中的技术点</title>
<link href="https://catinsides.github.io/2024/04/01/%E8%AE%B0%E4%B8%80%E6%AC%A1%E5%BD%95%E9%9F%B3%E8%BD%AC%E5%86%99%E9%A1%B9%E7%9B%AE%E7%BB%8F%E5%8E%86%E4%B8%AD%E7%9A%84%E6%8A%80%E6%9C%AF%E7%82%B9/"/>
<id>https://catinsides.github.io/2024/04/01/%E8%AE%B0%E4%B8%80%E6%AC%A1%E5%BD%95%E9%9F%B3%E8%BD%AC%E5%86%99%E9%A1%B9%E7%9B%AE%E7%BB%8F%E5%8E%86%E4%B8%AD%E7%9A%84%E6%8A%80%E6%9C%AF%E7%82%B9/</id>
<published>2024-04-01T09:43:02.000Z</published>
<updated>2024-04-02T05:26:02.543Z</updated>
<content type="html"><![CDATA[<p>最近做了一个录音转写的项目，其中有一些自认为有意思的技术点，在此记录一下。</p><h2 id="实时通话中的录音推流"><a href="#实时通话中的录音推流" class="headerlink" title="实时通话中的录音推流"></a>实时通话中的录音推流</h2><p>此次项目中的部分场景是，坐席与客户通话时，要将通话的内容在网页端进行展示。如果是通话完成后，将对话内容展示出来，可以直接参照 Azure 的官方教程中的示例，<a href="https://learn.microsoft.com/en-us/azure/ai-services/speech-service/get-started-speech-to-text?tabs=macos,terminal&pivots=programming-language-javascript">链接</a>。教程中，默认的方法就是将整个文件推送上去，无脑CV即可。 </p><p>通话完成进行录音识别，语音转文字，可能会遇到的难点有： </p><h3 id="录音文件与转写服务不在同一个服务器"><a href="#录音文件与转写服务不在同一个服务器" class="headerlink" title="录音文件与转写服务不在同一个服务器"></a>录音文件与转写服务不在同一个服务器</h3><p>此时，需要在自身的服务端实现一个录音下载的 stream 接口。转写服务的客户端，调用这个接口之后，可以选择将文件暂存到本地，或者直接将下载流转到 azure 的转写客户端。关键代码如下：</p><p>服务端接口：</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// 获取录音文件</span></span><br><span class="line"><span class="keyword">const</span> <span class="title function_">getRecord</span> = (<span class="params">req, res</span>) => {</span><br><span class="line"> <span class="keyword">const</span> { recordId } = req.<span class="property">query</span>;</span><br><span class="line"> <span class="keyword">const</span> recordPath = path.<span class="title function_">resolve</span>(__dirname, <span class="string">'record'</span>, <span class="string">`<span class="subst">${recordId}</span>.wav`</span>);</span><br><span class="line"></span><br><span class="line"> res.<span class="title function_">writeHead</span>(<span class="number">200</span>, {</span><br><span class="line"> <span class="string">'Content-Type'</span>: <span class="string">'audio/wav'</span>,</span><br><span class="line"> <span class="string">'Content-Length'</span>: fs.<span class="title function_">statSync</span>(recordPath).<span class="property">size</span></span><br><span class="line"> });</span><br><span class="line"></span><br><span class="line"> fs.<span class="title function_">createReadStream</span>(recordPath).<span class="title function_">pipe</span>(res);</span><br><span class="line">};</span><br></pre></td></tr></table></figure><p>转写服务客户端，同时要考虑到链接会被重定向，主要使用以下两个函数： </p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">function</span> <span class="title function_">sendGetRequest</span>(<span class="params">url, redirectTimes = <span class="number">0</span></span>) {</span><br><span class="line"> redirectTimes += <span class="number">1</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">Promise</span>(<span class="function">(<span class="params">resolve, reject</span>) =></span> {</span><br><span class="line"> <span class="keyword">if</span> (redirectTimes > <span class="number">10</span>) {</span><br><span class="line"> <span class="keyword">return</span> <span class="title function_">reject</span>(<span class="keyword">new</span> <span class="title class_">Error</span>(<span class="string">`HTTP request Redirects exceeded, <span class="subst">${redirectTimes}</span>`</span>));</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> req = http.<span class="title function_">get</span>(url, { <span class="attr">timeout</span>: <span class="number">3000</span> }, <span class="function">(<span class="params">res</span>) =></span> {</span><br><span class="line"> <span class="keyword">if</span> (res.<span class="property">statusCode</span> === <span class="number">302</span>) {</span><br><span class="line"> <span class="keyword">const</span> redirectUrl = res.<span class="property">headers</span>.<span class="property">location</span>;</span><br><span class="line"> <span class="variable language_">console</span>.<span class="title function_">log</span>(<span class="string">'Received 302 Found, redirecting to:'</span>, redirectUrl);</span><br><span class="line"> <span class="keyword">return</span> <span class="title function_">resolve</span>(<span class="title function_">sendGetRequest</span>(redirectUrl));</span><br><span class="line"> } <span class="keyword">else</span> <span class="keyword">if</span> (res.<span class="property">statusCode</span> === <span class="number">200</span>) {</span><br><span class="line"> <span class="title function_">resolve</span>(res);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="title function_">reject</span>(<span class="keyword">new</span> <span class="title class_">Error</span>(<span class="string">`HTTP request failed with status code <span class="subst">${res.statusCode}</span>`</span>));</span><br><span class="line"> }</span><br><span class="line"> });</span><br><span class="line"></span><br><span class="line"> req.<span class="title function_">on</span>(<span class="string">'error'</span>, <span class="function">(<span class="params">err</span>) =></span> {</span><br><span class="line"> <span class="title function_">reject</span>(err);</span><br><span class="line"> });</span><br><span class="line"> });</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">async</span> <span class="keyword">function</span> <span class="title function_">getRemoteFile</span>(<span class="params">file</span>) {</span><br><span class="line"> <span class="keyword">let</span> url = <span class="keyword">new</span> <span class="title function_">URL</span>(<span class="string">`/some-api`</span>, host);</span><br><span class="line"> <span class="keyword">let</span> filePath = path.<span class="title function_">join</span>(__dirname, <span class="string">'暂存路径'</span>);</span><br><span class="line"> <span class="keyword">let</span> writeStream = fs.<span class="title function_">createWriteStream</span>(filePath);</span><br><span class="line"> <span class="keyword">let</span> stream = <span class="keyword">await</span> <span class="title function_">sendGetRequest</span>(url.<span class="title function_">toString</span>());</span><br><span class="line"></span><br><span class="line"> stream.<span class="title function_">pipe</span>(writeStream);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">new</span> <span class="title class_">Promise</span>(<span class="function">(<span class="params">resolve, reject</span>) =></span> {</span><br><span class="line"> writeStream.<span class="title function_">on</span>(<span class="string">'error'</span>, <span class="function">(<span class="params">err</span>) =></span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="title function_">reject</span>(err);</span><br><span class="line"> });</span><br><span class="line"></span><br><span class="line"> writeStream.<span class="title function_">on</span>(<span class="string">'close'</span>, <span class="function">() =></span> {</span><br><span class="line"> <span class="keyword">return</span> <span class="title function_">resolve</span>(filePath);</span><br><span class="line"> });</span><br><span class="line"> });</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>按照此<a href="https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-recognize-speech?pivots=programming-language-javascript#recognize-speech-from-an-in-memory-stream">链接</a>中的方法，封装出一个接收文件输入流的转写函数 <code>readStreamPush</code>，将上面的文件流送入即可。 </p><h3 id="录音文件增量读取"><a href="#录音文件增量读取" class="headerlink" title="录音文件增量读取"></a>录音文件增量读取</h3><p>因为是实时转写的，所以需要不断的读取在改变大小的文件。在node中，可以很方便的实现这一点。</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">let</span> watcher = fs.<span class="title function_">watch</span>(file, <span class="function">(<span class="params">eventType, filename</span>) =></span> {</span><br><span class="line"> <span class="keyword">if</span> (eventType === <span class="string">'change'</span>) {</span><br><span class="line"> <span class="variable language_">this</span>.<span class="title function_">readIncrementalData</span>(file, <span class="function">(<span class="params">err, buffer, done</span>) =></span> {</span><br><span class="line"> <span class="keyword">if</span> (err) {</span><br><span class="line"> <span class="variable language_">this</span>.<span class="title function_">emit</span>(<span class="string">'error'</span>, err);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (buffer) {</span><br><span class="line"> <span class="title function_">readStreamPush</span>(buffer);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">if</span> (err || done) {</span><br><span class="line"> <span class="variable language_">this</span>.<span class="title function_">emit</span>(<span class="string">'end'</span>);</span><br><span class="line"></span><br><span class="line"> watcher.<span class="title function_">close</span>();</span><br><span class="line"> pushStream.<span class="title function_">close</span>();</span><br><span class="line"> <span class="title function_">readStreamPush</span>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (timeout) {</span><br><span class="line"> <span class="built_in">clearTimeout</span>(timeout);</span><br><span class="line"> <span class="keyword">return</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (timeout) {</span><br><span class="line"> <span class="built_in">clearTimeout</span>(timeout);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> timeout = <span class="built_in">setTimeout</span>(<span class="function">() =></span> {</span><br><span class="line"> watcher.<span class="title function_">close</span>();</span><br><span class="line"> pushStream.<span class="title function_">close</span>();</span><br><span class="line"> <span class="title function_">readStreamPush</span>();</span><br><span class="line"> }, <span class="number">60</span> * <span class="number">1000</span>);</span><br><span class="line"> });</span><br><span class="line"> }</span><br><span class="line">});</span><br><span class="line"></span><br><span class="line"><span class="keyword">let</span> lastPosition = <span class="number">0</span>; <span class="comment">// 记录上次读取的位置</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">function</span> <span class="title function_">readIncrementalData</span>(<span class="params">filePath, callback</span>) {</span><br><span class="line"> fs.<span class="title function_">open</span>(filePath, <span class="string">'r'</span>, <span class="function">(<span class="params">err, fd</span>) =></span> {</span><br><span class="line"> <span class="keyword">if</span> (err) {</span><br><span class="line"> <span class="keyword">return</span> <span class="title function_">callback</span>(err);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="comment">// 定位到上次读取的位置</span></span><br><span class="line"> fs.<span class="title function_">fstat</span>(fd, <span class="function">(<span class="params">err, stats</span>) =></span> {</span><br><span class="line"> <span class="keyword">if</span> (err) {</span><br><span class="line"> <span class="keyword">return</span> <span class="title function_">callback</span>(err);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">const</span> currentPosition = stats.<span class="property">size</span>;</span><br><span class="line"> <span class="keyword">const</span> bufferLength = currentPosition - lastPosition;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> (bufferLength > <span class="number">0</span>) {</span><br><span class="line"> <span class="keyword">const</span> buffer = <span class="title class_">Buffer</span>.<span class="title function_">alloc</span>(bufferLength);</span><br><span class="line"> fs.<span class="title function_">read</span>(fd, buffer, <span class="number">0</span>, bufferLength, lastPosition, <span class="function">(<span class="params">err, bytesRead, buffer</span>) =></span> {</span><br><span class="line"> <span class="keyword">if</span> (err) {</span><br><span class="line"> <span class="keyword">return</span> <span class="title function_">callback</span>(err);</span><br><span class="line"> }</span><br><span class="line"> <span class="comment">// 处理读取到的内容</span></span><br><span class="line"> <span class="title function_">callback</span>(<span class="literal">null</span>, buffer);</span><br><span class="line"></span><br><span class="line"> lastPosition = currentPosition; <span class="comment">// 更新上次读取的位置</span></span><br><span class="line"></span><br><span class="line"> fs.<span class="title function_">close</span>(fd);</span><br><span class="line"> });</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="title function_">callback</span>(<span class="literal">null</span>, <span class="literal">null</span>, <span class="literal">true</span>);</span><br><span class="line"> fs.<span class="title function_">close</span>(fd);</span><br><span class="line"> }</span><br><span class="line"> });</span><br><span class="line"> });</span><br><span class="line">}</span><br><span class="line"></span><br></pre></td></tr></table></figure><p>这里加了一个定时，如果一分钟后文件大小不再改变，则视为文件读取完毕。</p><h3 id="格式转换"><a href="#格式转换" class="headerlink" title="格式转换"></a>格式转换</h3><p>此时还会遇到的问题是，转写结果不准确，转写结果偏移量也不准确。经排查，推送的录音文件格式与 azure 要求的不符。 </p><p>要求是 <code>WAV audio files, including 16-bit, mono PCM, 8 kHz, and 16 kHz</code>. </p><p>那就还需要将上面的文件流进行格式转换，再送去推流。</p><p>关键代码如下: </p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">const</span> ffmpeg = <span class="built_in">require</span>(<span class="string">'fluent-ffmpeg'</span>);</span><br><span class="line"><span class="keyword">const</span> { <span class="title class_">Readable</span> } = <span class="built_in">require</span>(<span class="string">'stream'</span>);</span><br><span class="line"></span><br><span class="line"><span class="keyword">function</span> <span class="title function_">createConversionStream</span>(<span class="params">callback</span>) {</span><br><span class="line"> <span class="keyword">let</span> readStream = <span class="keyword">new</span> <span class="title class_">Readable</span>();</span><br><span class="line"> readStream.<span class="property">_read</span> = <span class="function">() =></span> { };</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">let</span> command = <span class="title function_">ffmpeg</span>()</span><br><span class="line"> .<span class="title function_">input</span>(readStream)</span><br><span class="line"> .<span class="title function_">audioFrequency</span>(<span class="number">16000</span>)</span><br><span class="line"> .<span class="title function_">audioChannels</span>(<span class="number">1</span>)</span><br><span class="line"> .<span class="title function_">audioCodec</span>(<span class="string">'pcm_s16le'</span>)</span><br><span class="line"> .<span class="title function_">on</span>(<span class="string">'error'</span>, <span class="function">(<span class="params">err</span>) =></span> {</span><br><span class="line"> <span class="title function_">callback</span>(err);</span><br><span class="line"> })</span><br><span class="line"> .<span class="title function_">on</span>(<span class="string">'end'</span>, <span class="function">() =></span> {</span><br><span class="line"> <span class="title function_">callback</span>(<span class="literal">null</span>, <span class="literal">null</span>);</span><br><span class="line"> })</span><br><span class="line"> .<span class="title function_">format</span>(<span class="string">'wav'</span>);</span><br><span class="line"></span><br><span class="line"> command</span><br><span class="line"> .<span class="title function_">pipe</span>()</span><br><span class="line"> .<span class="title function_">on</span>(<span class="string">'data'</span>, <span class="function"><span class="params">data</span> =></span> {</span><br><span class="line"> <span class="title function_">callback</span>(<span class="literal">null</span>, data);</span><br><span class="line"> });</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">function</span> (<span class="params">buffer</span>) {</span><br><span class="line"> <span class="keyword">if</span> (buffer) {</span><br><span class="line"> readStream.<span class="title function_">push</span>(buffer);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">if</span> (command && <span class="keyword">typeof</span> command.<span class="property">kill</span> === <span class="string">'function'</span>) {</span><br><span class="line"> command.<span class="title function_">kill</span>();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> };</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="variable language_">module</span>.<span class="property">exports</span> = {</span><br><span class="line"> createConversionStream,</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>上面的代码可视作返回输出流，直接送入封装的转写服务即可。</p><h2 id="转写结果展示"><a href="#转写结果展示" class="headerlink" title="转写结果展示"></a>转写结果展示</h2><p>在页面展示时，实现了一个类似于微信对话框的样式，可以直接用于展示转写结果，一左一右，坐席与客户。因为项目中有知识库，如果对话中涉及到了知识库设置的关键词，需要将关键词高亮显示，同时点击时要弹出知识库的内容。此处参考了类似于B站评论区的关键词的高亮样式。词语变色，右上角加了放大镜。 </p><p>将转写结果存入 elasticsearch 中，可以很方便的实现这一点。 </p><p>首先在前端实现一个自定义标签<code><search-keywords></code>，完成样式和点击的功能。 </p><p>在实时转写的查询中，加入知识库的关键字作为匹配条件，使用 elasticsearch 的 match_phrase 查询。给匹配到的关键词加上标签，返回到前端时就能渲染了。 </p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">query.<span class="property">body</span>.<span class="property">highlight</span> = {</span><br><span class="line"> <span class="string">"fields"</span>: {</span><br><span class="line"> [highlightField]: {</span><br><span class="line"> <span class="string">"type"</span>: <span class="string">"unified"</span>,</span><br><span class="line"> <span class="string">"number_of_fragments"</span>: <span class="number">0</span></span><br><span class="line"> }</span><br><span class="line"> },</span><br><span class="line"> <span class="string">"pre_tags"</span>: [<span class="string">"<search-keywords>"</span>],</span><br><span class="line"> <span class="string">"post_tags"</span>: [<span class="string">"</search-keywords>"</span>],</span><br><span class="line"> <span class="string">"require_field_match"</span>: <span class="literal">true</span>,</span><br><span class="line"> <span class="string">"encoder"</span>: <span class="string">"html"</span></span><br><span class="line">};</span><br></pre></td></tr></table></figure><p>本文为<a href="https://github.com/Catinsides">个人原创</a>，转载请署名且注明出处。</p>]]></content>
<summary type="html"><p>最近做了一个录音转写的项目，其中有一些自认为有意思的技术点，在此记录一下。</p>
<h2 id="实时通话中的录音推流"><a href="#实时通话中的录音推流" class="headerlink" title="实时通话中的录音推流"></a>实时通话中的录音推流</h2></summary>
<category term="Javascript" scheme="https://catinsides.github.io/tags/Javascript/"/>
</entry>
<entry>
<title>Hello Again</title>
<link href="https://catinsides.github.io/2024/03/28/Hello-Again/"/>
<id>https://catinsides.github.io/2024/03/28/Hello-Again/</id>
<published>2024-03-28T02:39:28.000Z</published>
<updated>2024-03-28T02:44:42.366Z</updated>
<content type="html"><![CDATA[<p>时隔多年，终于把废弃的博客救回来了。 </p><p>找了个简洁的博客主题，不搞花里胡哨。 </p><p>争取以后有时间多更新些新的内容上来。 </p><p>:)</p>]]></content>
<summary type="html"><p>时隔多年，终于把废弃的博客救回来了。 </p>
<p>找了个简洁的博客主题，不搞花里胡哨。 </p></summary>
<category term="Hello world" scheme="https://catinsides.github.io/tags/Hello-world/"/>
</entry>
<entry>
<title>关于拓展NodeMediaServer以支持JT1078</title>
<link href="https://catinsides.github.io/2019/07/17/%E5%85%B3%E4%BA%8E%E6%8B%93%E5%B1%95NodeMediaServer%E4%BB%A5%E6%94%AF%E6%8C%81JT1078/"/>
<id>https://catinsides.github.io/2019/07/17/%E5%85%B3%E4%BA%8E%E6%8B%93%E5%B1%95NodeMediaServer%E4%BB%A5%E6%94%AF%E6%8C%81JT1078/</id>
<published>2019-07-16T17:23:50.000Z</published>
<updated>2024-08-14T08:42:02.743Z</updated>
<content type="html"><![CDATA[<h3 id="Jul-13-2019"><a href="#Jul-13-2019" class="headerlink" title="Jul. 13, 2019"></a>Jul. 13, 2019</h3><p>可能与最近几月坚持跑步有关，头脑突然灵活了很多。</p><p>前几日闲来无事摸鱼时，突然就翻回到了NodeMediaServer（NMS）的代码。从服务启动，到视频的解析推流，大致三到四个文件，脉络清晰。NMS的文件结构给了我很大的启发。我突然就想到，完全可以按照NMS的结构，把JT1078的解析也整合进去。</p><p>想到去年年底，第一次接触JT1078，视频拉流推流这些概念的时候，完全是一脸懵比。当时准备在NMS上改造，也是在视频推流之后，在NMS解析视频包的位置进行后续处理。甚至研究了一段时间的RTMP握手，由于设备推流不是RTMP协议，在代码里还要判断，如果是设备推流过来的，要跳过RTMP握手阶段，再推流出去。现在回想起来，甚是痛苦。好在现在知道了如何使用FFmpeg作为推流工具，使得这一功能实现。</p><p>今日，我仿照NMS的结构，和前几篇文章提到的FFmpeg命令，主要以FFmpeg推流实现的JT1078解析代码推到了仓库里。明显这不是最优的方案。其中一个很大的问题就是一个通道就要启动三个子进程（视频，音频和合并音视频），一般每台设备会有六个通道，观看一个设备的视频直播时就要启动大量的FFmpeg，相当耗费资源。而且这些子进程管理起来比较麻烦，有可能随时某一个在收不到数据时的进程关闭，就会出现 <strong>EPIPE</strong> 错误。而我理想中的最佳方案并不是这样的。最初，我尝试将设备推流过来的视频和音频数据，分别使用NMS原代码中的RTMP封包函数处理，送到播放者的socket中去，但是并没有成功。尝试几轮，服务端没有报错，看起来成功了，但是播放端并没有数据。未找到原因在哪，就暂时转换思路，使用FFmpeg来实现了。</p><p>后来又想到，直接使用NMS的封包函数是不行的。视频数据倒是不需要额外处理，音频数据是G7xx格式的，RTMP并不支持，所以需要转码。看来代码层面的封包和解码还需要研究一段时间。</p><p>唉！现在是不能够用代码直接封装音视频包，只得使用FFmpeg命令。我要是懂得音视频解封装，还能吃这个亏？</p><h3 id="Jul-16-2019"><a href="#Jul-16-2019" class="headerlink" title="Jul. 16, 2019"></a>Jul. 16, 2019</h3><p>今日，研究了一下FFmpeg的命令，想到可以使用两条输入流（视频和音频）转化成一条输出流（RTMP）的方式实现转码和推流。这样就能够不创造出上一方案中多出来的视频和音频子进程了。在将设备的视频和音频流数据，分别存储下来后，使用FFmpeg命令推流</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ffmpeg -i test.264 -i test.mp3 -map 0:v -map 1:a -c:v copy -c:a copy -f flv rtmp://xxxx:1935/live/stream_xxxx</span><br></pre></td></tr></table></figure><p>命令执行成功，证明了这种方案可能行。于是，我将代码改造为，先将数据缓存，然后使用FFmpeg推流。谁知，实际情况是，缓存数据太小，FFmpeg进程启动后，瞬间就完成了推流，然后关闭了进程。造成的现象就是程序完全没有在（持续）推流。如果FFmpeg随时都要启动的话，有可能会遇到，总将文件从头开始推流的问题。遂暂时搁置了这种方案。</p><p>根据上一方案存在的问题，又想到了启动子进程后，可以使用写入 <strong>stdin</strong> 的方式，将数据写入FFmpeg的进程中。但是并没有找到NodeJs写入两条输入流到一个子进程中去的方法。就算是将视频流利用管道符串接到音频流的转码进程中，还是需要有两条输入流。看来这种方案暂时也不行了。</p><p>无意中翻到了 <strong>fluent-ffmpeg</strong> 这个库，好像有支持两条输入流的接口，待有时间再测试。</p><h3 id="Dec-31-2019"><a href="#Dec-31-2019" class="headerlink" title="Dec. 31, 2019"></a>Dec. 31, 2019</h3><p>万万没想到，在2020以前我能将上面的问题解决。 <del>先忘记fluent-ffmpeg这件事吧</del><br>注: <strong>以下内容适用于Linux系统</strong><br>解决方案仍然是，将视频和音频分流后存储为文件，不同的是文件类别为pipe文件，即命名管道文件(Named Pipe).它遵循先进先出原则(FIFO)，可以像普通文件一样管理。<br>可以使用以下命令创建管道文件:</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ mkfifo /tmp/one.pipe</span><br><span class="line">$ mkfifo /tmp/two.pipe</span><br></pre></td></tr></table></figure><p>然后将程序分离出的视频数据和音频数据像写文件一样写入即可，ffmpeg的进程则从这两个文件中读取数据，再转换成rtmp推流发送出去。<br>ffmpeg命令如下:</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ ffmpeg -loglevel panic -probesize 32 -re -r 16 -f h264 -i /tmp/one.pipe -f mp3 -i /tmp/two.pipe -map 0:v -map 1:a -c:v copy -c:a aac -strict -2 -preset ultrafast -tune zerolatency -f flv rtmp://xxxx:1935/live/stream_xxxx</span><br></pre></td></tr></table></figure><p>读入的文件需要指定格式，如h264和mp3(在上一步中将g726转为了mp3).<br>具体代码实现见我此次提交 <a href="https://github.com/Catinsides/Node-Media-Server/commit/17be2385e0bc00e4514c504035155d3703299947?diff=split">commit</a></p>]]></content>
<summary type="html">记录一些拓展过程中的想法.</summary>
<category term="笔记" scheme="https://catinsides.github.io/tags/%E7%AC%94%E8%AE%B0/"/>
</entry>
<entry>
<title>JT1078协议开发之双向对讲后记</title>
<link href="https://catinsides.github.io/2019/03/29/JT1078%E5%8D%8F%E8%AE%AE%E5%BC%80%E5%8F%91%E4%B9%8B%E5%8F%8C%E5%90%91%E5%AF%B9%E8%AE%B2%E5%90%8E%E8%AE%B0/"/>
<id>https://catinsides.github.io/2019/03/29/JT1078%E5%8D%8F%E8%AE%AE%E5%BC%80%E5%8F%91%E4%B9%8B%E5%8F%8C%E5%90%91%E5%AF%B9%E8%AE%B2%E5%90%8E%E8%AE%B0/</id>
<published>2019-03-28T16:00:00.000Z</published>
<updated>2024-08-14T08:42:02.739Z</updated>
<content type="html"><![CDATA[<h3 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h3><p>上一篇大致介绍了一下实时视频的开发流程，这一篇所要介绍的双向对讲算是上一篇内容的小进阶。由 JT/T 1078协议（以下简称1078）中介绍可知，双向对讲的数据传输方式也是在 <strong>5.5.3</strong> 表19中定义的。下达双向对讲指令通过改变修改 <strong>0x9101</strong> 消息ID中的参数来实现，本篇不再赘述。功能实现思路与上一篇相同，使用 <strong>nodejs</strong> 和 <strong>FFmpeg</strong> 实现。测试系统环境为 <strong>Linux</strong> .</p><h3 id="准备"><a href="#准备" class="headerlink" title="准备"></a>准备</h3><p>见上一篇</p><h3 id="流程"><a href="#流程" class="headerlink" title="流程"></a>流程</h3><p>实时视频流程大致如下：</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">participant 网页</span><br><span class="line">participant 服务器</span><br><span class="line">participant 设备</span><br><span class="line"></span><br><span class="line">网页 ->> 设备: 下达双向对讲指令</span><br><span class="line">设备 ->> 服务器: 推送音频数据</span><br><span class="line">服务器 ->> 网页: 转换音频格式推流至网页，并通知开启网页麦克风</span><br><span class="line">网页 ->> 服务器: 采集音频数据并推送</span><br><span class="line">服务器 ->> 设备: 将网页端音频转码，发送至设备</span><br></pre></td></tr></table></figure><h3 id="网页采集音频"><a href="#网页采集音频" class="headerlink" title="网页采集音频"></a>网页采集音频</h3><p>浏览器利用麦克风采集麦克风的方法，网上的代码有很多，方法也是如出一辙。<br>下面直接贴出代码：</p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> leftchannel = [];</span><br><span class="line"><span class="keyword">var</span> rightchannel = [];</span><br><span class="line"><span class="keyword">var</span> recorder = <span class="literal">null</span>;</span><br><span class="line"><span class="keyword">var</span> recording = <span class="literal">false</span>;</span><br><span class="line"><span class="keyword">var</span> recordingLength = <span class="number">0</span>;</span><br><span class="line"><span class="keyword">var</span> volume = <span class="literal">null</span>;</span><br><span class="line"><span class="keyword">var</span> audioInput = <span class="literal">null</span>;</span><br><span class="line"><span class="keyword">var</span> sampleRate = <span class="literal">null</span>;</span><br><span class="line"><span class="keyword">var</span> audioContext = <span class="literal">null</span>;</span><br><span class="line"><span class="keyword">var</span> context = <span class="literal">null</span>;</span><br><span class="line"><span class="keyword">var</span> outputElement = <span class="variable language_">document</span>.<span class="title function_">getElementById</span>(<span class="string">'output'</span>); <span class="comment">// 用于提示当前录音状态</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> (!navigator.<span class="property">getUserMedia</span>)</span><br><span class="line"> navigator.<span class="property">getUserMedia</span> = navigator.<span class="property">getUserMedia</span> || navigator.<span class="property">webkitGetUserMedia</span> ||</span><br><span class="line"> navigator.<span class="property">mozGetUserMedia</span> || navigator.<span class="property">msGetUserMedia</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> (navigator.<span class="property">getUserMedia</span>) {</span><br><span class="line"> navigator.<span class="title function_">getUserMedia</span>({ <span class="attr">audio</span>: <span class="literal">true</span> }, success, <span class="keyword">function</span> (<span class="params">e</span>) {</span><br><span class="line"> <span class="title function_">alert</span>(<span class="string">'Error capturing audio.'</span>);</span><br><span class="line"> });</span><br><span class="line">} <span class="keyword">else</span> {</span><br><span class="line"> <span class="title function_">alert</span>(<span class="string">'getUserMedia not supported in this browser.'</span>);</span><br><span class="line">};</span><br><span class="line"></span><br><span class="line"><span class="keyword">function</span> <span class="title function_">interleave</span>(<span class="params">leftChannel, rightChannel</span>) {</span><br><span class="line"> <span class="keyword">var</span> length = leftChannel.<span class="property">length</span> + rightChannel.<span class="property">length</span>;</span><br><span class="line"> <span class="keyword">var</span> result = <span class="keyword">new</span> <span class="title class_">Float32Array</span>(length);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">var</span> inputIndex = <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">var</span> index = <span class="number">0</span>; index < length;) {</span><br><span class="line"> result[index++] = leftChannel[inputIndex];</span><br><span class="line"> result[index++] = rightChannel[inputIndex];</span><br><span class="line"> inputIndex++;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> result;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">function</span> <span class="title function_">mergeBuffers</span>(<span class="params">channelBuffer, recordingLength</span>) {</span><br><span class="line"> <span class="keyword">var</span> result = <span class="keyword">new</span> <span class="title class_">Float32Array</span>(recordingLength);</span><br><span class="line"> <span class="keyword">var</span> offset = <span class="number">0</span>;</span><br><span class="line"> <span class="keyword">var</span> lng = channelBuffer.<span class="property">length</span>;</span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">var</span> i = <span class="number">0</span>; i < lng; i++) {</span><br><span class="line"> <span class="keyword">var</span> buffer = channelBuffer[i];</span><br><span class="line"> result.<span class="title function_">set</span>(buffer, offset);</span><br><span class="line"> offset += buffer.<span class="property">length</span>;</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> result;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">function</span> <span class="title function_">writeUTFBytes</span>(<span class="params">view, offset, string</span>) {</span><br><span class="line"> <span class="keyword">var</span> lng = string.<span class="property">length</span>;</span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">var</span> i = <span class="number">0</span>; i < lng; i++) {</span><br><span class="line"> view.<span class="title function_">setUint8</span>(offset + i, string.<span class="title function_">charCodeAt</span>(i));</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">function</span> <span class="title function_">success</span>(<span class="params">e</span>) {</span><br><span class="line"> <span class="variable language_">console</span>.<span class="title function_">log</span>(<span class="string">'Record Init Succcess!'</span>);</span><br><span class="line"></span><br><span class="line"> audioContext = <span class="variable language_">window</span>.<span class="property">AudioContext</span> || <span class="variable language_">window</span>.<span class="property">webkitAudioContext</span>;</span><br><span class="line"> context = <span class="keyword">new</span> <span class="title function_">audioContext</span>();</span><br><span class="line"> sampleRate = context.<span class="property">sampleRate</span>; <span class="comment">// 44100</span></span><br><span class="line"> volume = context.<span class="title function_">createGain</span>();</span><br><span class="line"> audioInput = context.<span class="title function_">createMediaStreamSource</span>(e);</span><br><span class="line"></span><br><span class="line"> audioInput.<span class="title function_">connect</span>(volume);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">var</span> bufferSize = <span class="number">2048</span>;</span><br><span class="line"> recorder = context.<span class="title function_">createScriptProcessor</span>(bufferSize, <span class="number">2</span>, <span class="number">2</span>);</span><br><span class="line"></span><br><span class="line"> recorder.<span class="property">onaudioprocess</span> = <span class="keyword">function</span> (<span class="params">e</span>) {</span><br><span class="line"> <span class="keyword">if</span> (!recording) <span class="keyword">return</span>;</span><br><span class="line"> <span class="keyword">var</span> left = e.<span class="property">inputBuffer</span>.<span class="title function_">getChannelData</span>(<span class="number">0</span>);</span><br><span class="line"> <span class="keyword">var</span> right = e.<span class="property">inputBuffer</span>.<span class="title function_">getChannelData</span>(<span class="number">1</span>);</span><br><span class="line"> leftchannel.<span class="title function_">push</span>(<span class="keyword">new</span> <span class="title class_">Float32Array</span>(left));</span><br><span class="line"> rightchannel.<span class="title function_">push</span>(<span class="keyword">new</span> <span class="title class_">Float32Array</span>(right));</span><br><span class="line"> recordingLength += bufferSize;</span><br><span class="line"> <span class="variable language_">console</span>.<span class="title function_">log</span>(<span class="string">'recording...'</span>);</span><br><span class="line"> <span class="title function_">combineAudio</span>([<span class="keyword">new</span> <span class="title class_">Float32Array</span>(left)], [<span class="keyword">new</span> <span class="title class_">Float32Array</span>(right)], bufferSize, sampleRate);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> volume.<span class="title function_">connect</span>(recorder);</span><br><span class="line"> recorder.<span class="title function_">connect</span>(context.<span class="property">destination</span>);</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">function</span> <span class="title function_">combineAudio</span>(<span class="params">leftc, rightc, bfSize, sampleRate</span>) {</span><br><span class="line"> <span class="keyword">var</span> leftBuffer = <span class="title function_">mergeBuffers</span>(leftc, bfSize);</span><br><span class="line"> <span class="keyword">var</span> rightBuffer = <span class="title function_">mergeBuffers</span>(rightc, bfSize);</span><br><span class="line"> <span class="keyword">var</span> interleaved = <span class="title function_">interleave</span>(leftBuffer, rightBuffer);</span><br><span class="line"> <span class="keyword">var</span> dataLen = interleaved.<span class="property">length</span> * <span class="number">2</span>;</span><br><span class="line"> <span class="keyword">var</span> buffer, view, index = <span class="number">0</span>;</span><br><span class="line"></span><br><span class="line"> buffer = <span class="keyword">new</span> <span class="title class_">ArrayBuffer</span>(dataLen); </span><br><span class="line"> view = <span class="keyword">new</span> <span class="title class_">DataView</span>(buffer);</span><br><span class="line"></span><br><span class="line"> <span class="keyword">var</span> lng = interleaved.<span class="property">length</span>;</span><br><span class="line"> <span class="keyword">var</span> volume = <span class="number">1</span>;</span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">var</span> i = <span class="number">0</span>; i < lng; i++) {</span><br><span class="line"> view.<span class="title function_">setInt16</span>(index, interleaved[i] * (<span class="number">0x7FFF</span> * volume), <span class="literal">true</span>);</span><br><span class="line"> index += <span class="number">2</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">var</span> blob = <span class="keyword">new</span> <span class="title class_">Blob</span>([view], { <span class="attr">type</span>: <span class="string">'audio/wave'</span> }); <span class="comment">// 最终音频数据</span></span><br><span class="line"> <span class="comment">// 将blob写入 websocket, 略</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>上述代码主要思路为通过改变全局变量 <strong>recording</strong> 控制录音，这个可以使用 <strong>button</strong> 元素实现。开启录音后，浏览器采集左右声道的音频数据，将数据存入Buffer后，合并两通道数据然后通过 <strong>DataView</strong> 和 <strong>Blob</strong> 包装后的数据发送至服务器。为了保证实时性，我这里使用的是 <strong>websocket</strong> 发送至后台。上述代码得到的音频数据为 <strong>PCMS16LE</strong> 原始数据，默认采样率为44.1k，所以体积比较巨大。可以通过减少声道数量和降低采样率，或者直接在浏览器端重新编码来减少体积。</p><h3 id="音频转码"><a href="#音频转码" class="headerlink" title="音频转码"></a>音频转码</h3><p>音频转码涉及两个部分，一是将网页端采集的音频数据转换成设备支持的格式发送给设备；二是将设备端发送的音频数据转成网页端支持的格式进行播放。<br>设备支持什么格式呢？由于设备的不同，可能初始默认的音频格式都不尽相同。最方便的肯定是直接操作设备进行查看了。还可以在下达双向对讲指令后，解析RTP包参照1078 <strong>表12</strong> 进行查看。我手里这台设备的音频格式是 <strong>G726LE</strong>.<br>首先进行网页端播放设备音频流的实现。有了上一篇文章的经验，音视频服务器和网页端播放器都是准备好的，一下子省去了很多的工作，只需要将设备音频数据转码发送给音视频服务器即可。然后，网页端从音视频服务器提供的地址拉流就能进行播放了。看起来很简单有木有，突然有点小兴奋。<br>接下来的思路与视频处理相似，使用子进程开启 <strong>FFmpeg</strong> 进行转码并推流。由于推流要使用 <strong>flv</strong> 格式的 <strong>RTMP</strong> 流，而 <strong>flv</strong> 支持的音频格式是有限的（<a href="https://trac.ffmpeg.org/wiki/SupportedMediaTypesInFormats">点击查看</a>），光是进行音频转码就占用了一个FFmpeg子进程，所以还需要另一个进程将转码后的数据推流出去。听起来很麻烦，但FFmpeg支持数据流处理，所以实现起来非常简单，只需要将两个子进程的STDIN, STDOUT pipe 起来就行了，代码如下：</p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">const</span> { spawn } = <span class="built_in">require</span>(<span class="string">'child_process'</span>);</span><br><span class="line"><span class="keyword">const</span> <span class="variable constant_">FFMPEG</span> = <span class="string">'/usr/bin/ffmpeg'</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">var</span> cmds1 = [</span><br><span class="line"> <span class="string">'-loglevel'</span>, <span class="string">'panic'</span>,</span><br><span class="line"> <span class="string">'-probesize'</span>, <span class="string">'32'</span>,</span><br><span class="line"> <span class="string">'-f'</span>, <span class="string">'g726le'</span>,</span><br><span class="line"> <span class="string">'-code_size'</span>, <span class="string">'5'</span>,</span><br><span class="line"> <span class="string">'-ar'</span>, <span class="string">'8000'</span>,</span><br><span class="line"> <span class="string">'-ac'</span>, <span class="string">'1'</span>,</span><br><span class="line"> <span class="string">'-i'</span>, <span class="string">'-'</span>,</span><br><span class="line"> <span class="string">'-ar'</span>, <span class="string">'44100'</span>,</span><br><span class="line"> <span class="string">'-acodec'</span>, <span class="string">'libmp3lame'</span>,</span><br><span class="line"> <span class="string">'-f'</span>, <span class="string">'mp3'</span>,</span><br><span class="line"> <span class="string">'pipe:1'</span></span><br><span class="line">];</span><br><span class="line"><span class="keyword">var</span> cmds2 = [</span><br><span class="line"> <span class="string">'-i'</span>, <span class="string">'-'</span>,</span><br><span class="line"> <span class="string">'-vn'</span>,</span><br><span class="line"> <span class="string">'-c'</span>, <span class="string">'copy'</span>,</span><br><span class="line"> <span class="string">'-f'</span>, <span class="string">'flv'</span>,</span><br><span class="line"> rtmpUrl</span><br><span class="line">];</span><br><span class="line"></span><br><span class="line"><span class="keyword">var</span> child1 = <span class="title function_">spawn</span>(<span class="variable constant_">FFMPEG</span>, cmds1);</span><br><span class="line"><span class="keyword">var</span> child2 = <span class="title function_">spawn</span>(<span class="variable constant_">FFMPEG</span>, cmds2);</span><br><span class="line"></span><br><span class="line">child1.<span class="property">stdin</span>.<span class="title function_">write</span>(data);</span><br><span class="line">child1.<span class="property">stdout</span>.<span class="title function_">pipe</span>(child2.<span class="property">stdin</span>);</span><br></pre></td></tr></table></figure><p>这样，网页端播放器就能使用 <strong>RTMP</strong> 链接播放设备的音频数据了。<br>回来处理网页端到设备端的部分，思路也清晰了起来，只需将音频数据写到设备就行了。已经知道了设备支持的编码格式是 <strong>G726LE</strong>，所以首先需要将音频转码：</p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">const</span> { spawn } = <span class="built_in">require</span>(<span class="string">'child_process'</span>);</span><br><span class="line"><span class="keyword">const</span> <span class="variable constant_">FFMPEG</span> = <span class="string">'/usr/bin/ffmpeg'</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">var</span> cmds = [</span><br><span class="line"> <span class="string">'-loglevel'</span>, <span class="string">'panic'</span>,</span><br><span class="line"> <span class="string">'-f'</span>, <span class="string">'s16le'</span>,</span><br><span class="line"> <span class="string">'-ar'</span>, <span class="string">'44.1k'</span>,</span><br><span class="line"> <span class="string">'-ac'</span>, <span class="string">'2'</span>,</span><br><span class="line"> <span class="string">'-i'</span>, <span class="string">'-'</span>,</span><br><span class="line"> <span class="string">'-acodec'</span>, <span class="string">'g726le'</span>,</span><br><span class="line"> <span class="string">'-code_size'</span>, <span class="string">'5'</span>,</span><br><span class="line"> <span class="string">'-ar'</span>, <span class="string">'8k'</span>,</span><br><span class="line"> <span class="string">'-ac'</span>, <span class="string">'1'</span>,</span><br><span class="line"> <span class="string">'-f'</span>, <span class="string">'g726le'</span>,</span><br><span class="line"> <span class="string">'pipe:1'</span></span><br><span class="line">];</span><br><span class="line"><span class="keyword">var</span> _process = <span class="title function_">spawn</span>(<span class="variable constant_">FFMPEG</span>, cmds);</span><br></pre></td></tr></table></figure><p>开启子进程后，将网页端数据写入进程STDIN，再将STDOUT数据写入设备连至服务器的TCP连接即可。因为流程中，是先下达指令，设备连接上来后，再进行网页端数据写入，设备的连接是很容易获取到的。</p><h3 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h3><p>实时视频和双向对讲的实现思路大体上是一致，区别在FFmpeg的参数不同，多开启了几个进程而已。不由得感叹FFmpeg的强大。前后两篇文章中，涉及到业务的代码都是一笔带过，而主要列举了FFmpeg的命令。不要小看这简单的几行命令，对于从没接触过FFmpeg和音视频知识几乎为零的我来说，花费了大量的时间来搜索和尝试。相比之下，业务上码代码花费的时间更少。业务代码中也有很多值得注意的点，如设备连接和断开时，FFmpeg进程的管理；FFmpeg进程写入时，EPIPE的处理（管理各个子进程的事件回调）；双向对讲状态的共享（Redis实现）等等。经过这一项目的一番折腾，也算是了解一些音视频方面的知识，也对这方面有了一番兴趣。打算借着没有减退的热度开个音视频相关的小项目，希望自己不要弃坑。</p>]]></content>
<summary type="html">记录一下JT1078协议中双向对讲功能的开发流程。</summary>
<category term="JT1078" scheme="https://catinsides.github.io/tags/JT1078/"/>
<category term="FFmpeg" scheme="https://catinsides.github.io/tags/FFmpeg/"/>
</entry>
<entry>
<title>JT1078协议开发之实时视频后记</title>
<link href="https://catinsides.github.io/2019/03/16/JT1078%E5%8D%8F%E8%AE%AE%E5%BC%80%E5%8F%91%E4%B9%8B%E5%AE%9E%E6%97%B6%E8%A7%86%E9%A2%91%E5%90%8E%E8%AE%B0/"/>
<id>https://catinsides.github.io/2019/03/16/JT1078%E5%8D%8F%E8%AE%AE%E5%BC%80%E5%8F%91%E4%B9%8B%E5%AE%9E%E6%97%B6%E8%A7%86%E9%A2%91%E5%90%8E%E8%AE%B0/</id>
<published>2019-03-15T16:00:00.000Z</published>
<updated>2024-08-14T08:42:02.739Z</updated>
<content type="html"><![CDATA[<h3 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h3><p>前几个月一直在进行JT/T 1078协议（以下简称1078）相关项目的开发，其中涉及各种音视频协议和网络协议，还有音视频服务器和处理软件的知识。从一开始处于知识盲区的我，一路摸爬滚打，google和阅读书籍，挖坑填坑，总算是把项目需要的功能给研究出来了。当然，1078本身描述的功能很多，全部实现需要大量的人力和时间，一篇文章也不可能讲完。所以，打算用两篇文章主要记录一下实时视频和双向对讲的实现方法。这一篇为实时视频，双向对讲放在下一篇。功能皆使用 <strong>nodejs</strong> 和 <strong>FFmpeg</strong> 实现，系统环境为 <strong>Linux</strong> .</p><h3 id="准备"><a href="#准备" class="headerlink" title="准备"></a>准备</h3><p>项目需求简单地说就是，有一台支持1078协议的设备，接收特定指令后，通过协议内容将音视频数据发送至服务器（以下简称推流），服务器再将音视频数据转发出去，使得客户端或者网页播放器能够收看到设备发送的数据（以下简称拉流），以达到实时视频的效果。后面功能的实现皆为网页端操作。</p><p>所以至少要准备：</p><ul><li>1078协议文档</li><li>1078协议设备</li><li>音视频服务器</li><li>播放器</li></ul><h3 id="流程"><a href="#流程" class="headerlink" title="流程"></a>流程</h3><p>实时视频流程大致如下：</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">participant 网页</span><br><span class="line">participant 服务器</span><br><span class="line">participant 设备</span><br><span class="line"></span><br><span class="line">网页 ->> 设备: 下达推送视频指令</span><br><span class="line">设备 ->> 服务器: 推送视频数据</span><br><span class="line">服务器 ->> 网页: 转换为网页支持的格式</span><br></pre></td></tr></table></figure><h3 id="音视频服务器与解析"><a href="#音视频服务器与解析" class="headerlink" title="音视频服务器与解析"></a>音视频服务器与解析</h3><p>下达推送指令部分可按照协议要求的格式轻松搞定，主要问题在于如何将设备的音视频数据推流至服务器。服务器再转换为网页端能够拉流播放的格式。说到视频直播，推流拉流，首先想到的是RTMP。在搜索引擎中搜一搜，支持RTMP的音视频服务器，无论是开源还是闭源，都有很多。首先找到一个部署方便的，放在服务器上待命。<br>部署完成之后，可以使用FFmpeg推送本地文件至服务器测试好不好用。</p><p>FFmpeg命令如下：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ffmpeg -re -i localFile.mp4 -c copy -f flv rtmp://server:1935/live/streamName</span><br></pre></td></tr></table></figure><p>RTMP链接后部分可能根据服务器有所不同。端口一般为1935.<br>播放RTMP链接，可以使用FFmpeg内置的ffplay，或者其他在线播放网站即可。</p><p>由1078文档内容可知，设备的音视频传输方式，定义在 <strong>5.5.3</strong> 部分。</p><blockquote><p>实时音视频流数据的传输参考RTP协议，使用UDP或TCP承载。</p></blockquote><p>可知，设备推流并不是RTMP协议，而是魔改的RTP协议，具体内容在表19中。这和服务器接收的数据协议不同，所以需要将设备推流转换为服务器能够接收的协议格式。<br>第一次搞这种东西，只看文档一头雾水，没别的办法，搭建一个TCP服务，让设备把数据发过来，验证数据格式是否与文档中描述的一致。</p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">const</span> net = <span class="built_in">require</span>(<span class="string">'net'</span>);</span><br><span class="line"><span class="keyword">const</span> <span class="variable constant_">HOST</span> = <span class="string">'xx'</span>, <span class="variable constant_">PORT</span> = <span class="string">'xx'</span>; <span class="comment">// 在下发指令中指定</span></span><br><span class="line"><span class="keyword">const</span> server = net.<span class="title function_">createServer</span>(<span class="function"><span class="params">socket</span> =></span> {</span><br><span class="line"> <span class="variable language_">console</span>.<span class="title function_">log</span>(<span class="string">'TCP Server Created!'</span>);</span><br><span class="line"></span><br><span class="line"> socket.<span class="title function_">on</span>(<span class="string">'error'</span>, <span class="function"><span class="params">err</span> =></span> {</span><br><span class="line"> <span class="variable language_">console</span>.<span class="title function_">log</span>(<span class="string">'TCP Server Error =='</span>, err);</span><br><span class="line"> });</span><br><span class="line"></span><br><span class="line"> socket.<span class="title function_">on</span>(<span class="string">'end'</span>, <span class="function"><span class="params">_</span> =></span> {</span><br><span class="line"> <span class="variable language_">console</span>.<span class="title function_">log</span>(<span class="string">'TCP Server Closed.'</span>);</span><br><span class="line"> });</span><br><span class="line"></span><br><span class="line"> socket.<span class="title function_">setKeepAlive</span>(<span class="literal">true</span>, <span class="number">2000</span>);</span><br><span class="line"> socket.<span class="built_in">setTimeout</span>(<span class="number">10000</span>, <span class="function"><span class="params">_</span> =></span> {</span><br><span class="line"> socket.<span class="title function_">end</span>(<span class="string">'TCP Socket Broken.'</span>);</span><br><span class="line"> });</span><br><span class="line">});</span><br><span class="line">server.<span class="title function_">listen</span>(<span class="variable constant_">PORT</span>, <span class="variable constant_">HOST</span>, <span class="function"><span class="params">_</span> =></span> {</span><br><span class="line"> <span class="variable language_">console</span>.<span class="title function_">log</span>(<span class="string">`Server Start Successfully, Listen on <span class="subst">${PORT}</span>.`</span>);</span><br><span class="line">});</span><br><span class="line">server.<span class="title function_">on</span>(<span class="string">'connection'</span>, <span class="function"><span class="params">socket</span> =></span> {</span><br><span class="line"> <span class="keyword">var</span> { remoteAddress } = socket, { address, port } = socket.<span class="title function_">address</span>();</span><br><span class="line"> <span class="variable language_">console</span>.<span class="title function_">log</span>(<span class="string">'Address is '</span>, address, <span class="string">'. Port is'</span>, port, <span class="string">'. remoteAddress is '</span>, remoteAddress);</span><br><span class="line"></span><br><span class="line"> socket.<span class="title function_">on</span>(<span class="string">'data'</span>, <span class="function"><span class="params">data</span> =></span> {</span><br><span class="line"> <span class="comment">// console.log('TCP ON DATA');</span></span><br><span class="line"> });</span><br><span class="line"></span><br><span class="line"> socket.<span class="title function_">on</span>(<span class="string">'end'</span>, <span class="function"><span class="params">_</span> =></span> {</span><br><span class="line"> <span class="variable language_">console</span>.<span class="title function_">log</span>(<span class="string">'Bye'</span>);</span><br><span class="line"> });</span><br><span class="line">});</span><br></pre></td></tr></table></figure><p>设备推流后会触发socket的on data 事件，由于data是Buffer，所以需要把data转换成 <strong>16进制</strong> 的数据才能对照文档分析。转换完成后，可以大致看到一些规律，与文档中描述的一致。一个data变量可能包含多个以 <strong>0x30 0x31 0x63 0x64</strong> 开头的数据，对应文档中的帧头标识。有没有可能在负载数据中也出现帧头呢，那就再往后取一个字节，降低概率。可以观察到第4至第5字节，都是 <strong>0x81</strong> .具体含义可以看 <strong>RFC 3550</strong>.每个RTP包除了前30字节（也有可能少于30）的数据描述外，就是媒体数据payload了。<br>接下来要做的是，从TCP数据中分离出每个RTP包。TCP是基于流式传输的（我真的很讨厌“粘包”这个词），每个data不一定以帧头开始，或者结束，而且中间可能包含多个RTP包。但是已经知道了帧头一定是 <strong>0x30 0x31 0x63 0x64 0x81</strong> , 所以可以用帧头来切分TCP数据。再将切分出来的RTP包传入到下一个函数中单独处理。这部分处理很简单，不再贴代码，需要注意的地方就是如何将两个TCP data中的RTP包数据组合再传入函数。<br>先假定现在是最理想的情况，即不需要考虑RTP包乱序，视频帧乱序的情况，以这种前提将payload存储下来看看能否正常播放。当然，上面的再取一个 <strong>0x81</strong> 也是在这种前提下考虑的。<br>接下来就是要分析RTP包的数据了。完全对照文档中的表19按字节读取即可。网络传输使用的是 <strong>大端模式</strong> ，nodejs中使用的函数主要为以下几个：</p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">buf.<span class="title function_">slice</span>([start[, end]])</span><br><span class="line">buf.<span class="title function_">readInt8</span>(offset)</span><br><span class="line">buf.<span class="title function_">readInt16BE</span>(offset)</span><br><span class="line">buf.<span class="title function_">readInt32BE</span>(offset)</span><br></pre></td></tr></table></figure><p>解析完成后，可以看到当前RTP包所承载的数据信息。如包序号，SIM卡号，通道号，数据类型，分包标记等。原子包不做处理，分包需要将后续的包组合成一个数据。从第5字节中的数据可知，当前包负载中的媒体类型，解析后对应表12查得可知，音频格式为 <strong>G.726</strong>，视频格式为 <strong>H.264</strong>.不同设备的格式可能有所不同。将音频包和视频包分别存储为音频和视频文件，再使用软件播放验证前面解析程序的正确性。全部正确的话，这些文件都是可以正常播放的。<br>接下来就需要想办法如何把这些数据推流到音视频服务器了。服务器接收的格式为RTMP流，需要把从RTP分离出的媒体数据转化为RTMP流推到服务器。由于本人还不能直接用nodejs撸出一个转码和推流服务的程序出来，接下来的工作便交给了 <strong>FFmpeg</strong> 处理。<br>先来看一下如何处理视频数据，用FFmpeg将本地文件推送至RTMP服务器，查官方文档可知，得到下面的命令：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">ffmpeg -loglevel panic \</span><br><span class="line"> -probesize 32 \</span><br><span class="line"> -re \</span><br><span class="line"> -r 16 \</span><br><span class="line"> -i localfile.h264 \</span><br><span class="line"> -c:v libx264 \</span><br><span class="line"> -preset ultrafast \</span><br><span class="line"> -tune zerolatency \</span><br><span class="line"> -profile:v baseline \</span><br><span class="line"> -f flv <span class="string">"rtmp://xxx"</span></span><br></pre></td></tr></table></figure><p>上面这条语句不止能推送文件，还是最终的优化版。<br>解释一下部分选项：</p><ul><li>-loglevel panic 用于屏蔽ffmpeg的输出内容</li><li>-probesize 32 用于降低ffmpeg转换延迟</li><li>-r 16 设置帧率，需要与设备端一致，否则会出现播放速度太快的问题</li><li>-f flv “” 输出rtmp链接，这里需要使用SIM卡号和通道号组合成特定链接</li></ul><p>有了这条命令，就该想办法如何把RTP中的payload输入到ffmpeg了。<br>ffmpeg支持从STDIN输入的数据，所以只需要将ffmpeg执行命令放入nodejs的child process子进程中，然后把payload数据写入该子进程的STDIN即可，代码如下：</p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">const</span> { spawn } = <span class="built_in">require</span>(<span class="string">'child_process'</span>);</span><br><span class="line"><span class="keyword">const</span> <span class="variable constant_">FFMPEG</span> = <span class="string">'/usr/bin/ffmpeg'</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">var</span> cmds = [</span><br><span class="line"> <span class="string">'-loglevel'</span>, <span class="string">'panic'</span>,</span><br><span class="line"> <span class="string">'-probesize'</span>, <span class="string">'32'</span>,</span><br><span class="line"> <span class="string">'-re'</span>,</span><br><span class="line"> <span class="string">'-r'</span>, <span class="string">'16'</span>,</span><br><span class="line"> <span class="string">'-i'</span>, <span class="string">'-'</span>,</span><br><span class="line"> <span class="string">'-c:v'</span>, <span class="string">'libx264'</span>,</span><br><span class="line"> <span class="string">'-preset'</span>, <span class="string">'ultrafast'</span>,</span><br><span class="line"> <span class="string">'-tune'</span>, <span class="string">'zerolatency'</span>,</span><br><span class="line"> <span class="string">'-profile:v'</span>, <span class="string">'baseline'</span>,</span><br><span class="line"> <span class="string">'-f'</span>, <span class="string">'flv'</span>,</span><br><span class="line"> rtmpUrl</span><br><span class="line">];</span><br><span class="line"><span class="keyword">var</span> child = <span class="title function_">spawn</span>(<span class="variable constant_">FFMPEG</span>, cmds);</span><br><span class="line">child.<span class="property">stdin</span>.<span class="title function_">write</span>(data);</span><br></pre></td></tr></table></figure><p>区别在于，将本地文件的名称换成了 - .ffmpeg会在子进程中将payload数据转化成rtmp推流到服务器。完成这一部分的代码之后，可以先尝试着将数据推流到音视频服务器，然后找一个能在线看rtmp的网站进行测试。程序无误的情况下，可以看到直播数据。在单台设备全通道的情况下，可以做到延迟 <strong>2-3s</strong> .说明上面的“大胆”假设是正确的，一下子感觉轻松了不少:)</p><p>再接下来需要进一步完善程序，如：</p><ul><li>设备收到停止视频推送指令后，需要关闭ffmpeg的子进程</li><li>多设备，多通道情况下的ffmpeg子进程管理</li></ul><p>这就与1078协议开发无关了，考验写程序功底的时候到了，本文不再赘述。</p><p>音频部分涉及到转码，留在与下一篇双向对讲功能一起说。</p><h3 id="网页端"><a href="#网页端" class="headerlink" title="网页端"></a>网页端</h3><p>网页端无论是Flash还是H5的视频播放器有很多，翻翻文档API就可以查到如何播放指定链接的视频。如果音视频服务器不仅能提供rtmp流，还能提供flv流，网页播放的实现可以更加灵活。</p><h3 id="后记"><a href="#后记" class="headerlink" title="后记"></a>后记</h3><p>1078协议实时视频的开发，说难不难，说简单不简单。它难就难在网上的资料居然如此之少，完全需要靠自己摸索验证。而且在开发验证的过程中，很容易走弯路。开发完成后，回头再看，发现只是需要按文档读取数据而已，并没有涉及音视频领域更深一层的知识。<br>本文并没有完全使用nodejs进行转码和推送的实现，而是使用了ffmpeg作为子进程进行推流。恕本人学识有限，音视频知识的学习任重而道远，暂时只能以这样的“笨”方法实现功能需求。新的学习计划已提上日程，希望在以后的日子里能够进一步改进。</p>]]></content>
<summary type="html">记录一下JT1078协议中实时视频功能的开发流程。</summary>
<category term="笔记" scheme="https://catinsides.github.io/tags/%E7%AC%94%E8%AE%B0/"/>
<category term="JT1078" scheme="https://catinsides.github.io/tags/JT1078/"/>
<category term="FFmpeg" scheme="https://catinsides.github.io/tags/FFmpeg/"/>
</entry>
<entry>
<title>2018都做了些什么</title>
<link href="https://catinsides.github.io/2018/12/31/2018%E9%83%BD%E5%81%9A%E4%BA%86%E4%BA%9B%E4%BB%80%E4%B9%88/"/>
<id>https://catinsides.github.io/2018/12/31/2018%E9%83%BD%E5%81%9A%E4%BA%86%E4%BA%9B%E4%BB%80%E4%B9%88/</id>
<published>2018-12-31T15:59:59.000Z</published>
<updated>2024-08-14T08:42:02.736Z</updated>
<content type="html"><![CDATA[<blockquote><p>不要高估一天能做到的事，也不要低估一年能做到的事。</p></blockquote><p>我这个人是拒绝鸡汤的。偶然在V2里看到这句话，思考了一下，也确实有一些道理。回想到去年此时的自己，怎么也想不到会学习了这么多的新技能。每天学习一点点，日积月累，真的很重要。</p><p>回想这一年：</p><ul><li>印象最深的一本书，是<a href="https://book.douban.com/subject/27081224/">《学会提问》</a>。书本身很小巧，几个小时就能读完，但是收获的内容可以说是受益终生。作者从提问的意义，到提问分类，再到优质提问的方法，阐述了学会做优质的提问是多么的重要。问题也可分为抛给对方还是抛给自己的。抛给对方的问题，能够带来有益的思考，还能解决自己的迷惑。而抛给自己的优质提问是有力的内驱力。</li><li>印象最深的影视剧，是<a href="https://movie.douban.com/subject/26602304/">《重版出来！》</a>。有一点浮夸风的职场剧，毕竟是漫改。看罢，真是感觉热血到不行。也了解到了，一部漫画作品从创作到连载的艰辛。如果能够依靠自己的爱好赚钱，真是既痛苦又快乐的事情。以至于，看完剧的我买了几本《超级漫画素描技法》，然而并没有怎么看，放到书架上吃灰中（摊手）。</li></ul><p>回到上班摸鱼，除了几个从部署到完成业务的与深度学习相关的功能，还是在前端页面和后端增删改查之中打转。深度学习并没有深度学习，前后端相比以前熟悉了很多，四舍五入也算是进步了吧。在业余时间，学习了下docker的使用，这绝对是这一年最大的收获了。在部署服务和测试功能上，都有很大的帮助，绝对是早学早受益。</p><p>夏季之前，也尝试了下Unity3D.利用了半个月左右的下班后时间，制作出了极其简陋的第三人称射击游戏。游戏拥有基本的界面和操作，敌人不能动，但是能够随机地点重生。说的好像有点roguelike呢。深刻体会到了游戏制作的艰辛，由衷地钦佩能够包揽美术、音乐和策划的独立制作人。</p><p>这一年一直坚持下来的就是日语的学习了，受影视音乐文化的影响，也保持着高涨的热情。虽然进步缓慢，但督促自己能够坚持下去。年底接手了与音视频相关的项目，接触不少新知识，C/C++的学习也提上了日程(虽然被《C++ Primer Plus》的厚度吓到劝退)。希望在新的一年里，珍惜时间，努力学习，想要放弃的时候，想想为什么要开始。</p>]]></content>
<summary type="html">回忆我的2018.</summary>
<category term="笔记" scheme="https://catinsides.github.io/tags/%E7%AC%94%E8%AE%B0/"/>
</entry>
<entry>
<title>给console.log一点颜色看看</title>
<link href="https://catinsides.github.io/2017/07/03/%E7%BB%99console.log%E4%B8%80%E7%82%B9%E9%A2%9C%E8%89%B2%E7%9C%8B%E7%9C%8B/"/>
<id>https://catinsides.github.io/2017/07/03/%E7%BB%99console.log%E4%B8%80%E7%82%B9%E9%A2%9C%E8%89%B2%E7%9C%8B%E7%9C%8B/</id>
<published>2017-07-03T15:44:15.000Z</published>
<updated>2024-08-14T08:42:02.744Z</updated>
<content type="html"><![CDATA[<p>开发中少不了使用console.log进行调试，那么如何使打印出的内容具有颜色呢？</p><p>首先要说明的是，这里指的打印是在命令行中进行输出的，而不是在chrome控制台中。</p><p>答案是，把console.log()第一个参数设置为ANSI转义码即可, 第二个参数为需要打印的内容。</p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">// 像这样</span></span><br><span class="line"><span class="variable language_">console</span>.<span class="title function_">log</span>(<span class="string">"\x1b[31m"</span>, <span class="string">"I'm green."</span>);</span><br><span class="line"></span><br><span class="line"><span class="comment">// 当然，聪明而又懒惰的我们不会一个一个去敲这些字符，真是光看看就觉得麻烦啊，So...</span></span><br><span class="line"><span class="keyword">const</span> <span class="title class_">FgRed</span> = <span class="string">"\x1b[31m"</span>;</span><br><span class="line"><span class="variable language_">console</span>.<span class="title function_">log</span>(<span class="title class_">FgRed</span>, <span class="string">"I'm red."</span>);</span><br><span class="line"></span><br><span class="line"><span class="comment">// 真是雕虫小技啊:P</span></span><br><span class="line"><span class="comment">// 这里还有有更多的颜色</span></span><br><span class="line"><span class="title class_">Reset</span> = <span class="string">"\x1b[0m"</span>,</span><br><span class="line"><span class="title class_">Bright</span> = <span class="string">"\x1b[1m"</span>,</span><br><span class="line"><span class="title class_">Dim</span> = <span class="string">"\x1b[2m"</span>,</span><br><span class="line"><span class="title class_">Underscore</span> = <span class="string">"\x1b[4m"</span>,</span><br><span class="line"><span class="title class_">Blink</span> = <span class="string">"\x1b[5m"</span>,</span><br><span class="line"><span class="title class_">Reverse</span> = <span class="string">"\x1b[7m"</span>,</span><br><span class="line"><span class="title class_">Hidden</span> = <span class="string">"\x1b[8m"</span>,</span><br><span class="line"><span class="title class_">FgBlack</span> = <span class="string">"\x1b[30m"</span>,</span><br><span class="line"><span class="title class_">FgRed</span> = <span class="string">"\x1b[31m"</span>,</span><br><span class="line"><span class="title class_">FgGreen</span> = <span class="string">"\x1b[32m"</span>,</span><br><span class="line"><span class="title class_">FgYellow</span> = <span class="string">"\x1b[33m"</span>,</span><br><span class="line"><span class="title class_">FgBlue</span> = <span class="string">"\x1b[34m"</span>,</span><br><span class="line"><span class="title class_">FgMagenta</span> = <span class="string">"\x1b[35m"</span>,</span><br><span class="line"><span class="title class_">FgCyan</span> = <span class="string">"\x1b[36m"</span>,</span><br><span class="line"><span class="title class_">FgWhite</span> = <span class="string">"\x1b[37m"</span>,</span><br><span class="line"><span class="title class_">BgBlack</span> = <span class="string">"\x1b[40m"</span>,</span><br><span class="line"><span class="title class_">BgRed</span> = <span class="string">"\x1b[41m"</span>,</span><br><span class="line"><span class="title class_">BgGreen</span> = <span class="string">"\x1b[42m"</span>,</span><br><span class="line"><span class="title class_">BgYellow</span> = <span class="string">"\x1b[43m"</span>,</span><br><span class="line"><span class="title class_">BgBlue</span> = <span class="string">"\x1b[44m"</span>,</span><br><span class="line"><span class="title class_">BgMagenta</span> = <span class="string">"\x1b[45m"</span>,</span><br><span class="line"><span class="title class_">BgCyan</span> = <span class="string">"\x1b[46m"</span>,</span><br><span class="line"><span class="title class_">BgWhite</span> = <span class="string">"\x1b[47m"</span>;</span><br></pre></td></tr></table></figure><p>拓展阅读:<a href="https://en.wikipedia.org/wiki/ANSI_escape_code">ANSI escape code</a></p>]]></content>
<summary type="html">开发中少不了使用console.log进行调试，那么如何使打印出的内容具有颜色呢？</summary>
<category term="Javascript" scheme="https://catinsides.github.io/tags/Javascript/"/>
</entry>
<entry>
<title>JS野生程序员进修指南</title>
<link href="https://catinsides.github.io/2017/05/15/JS%E9%87%8E%E7%94%9F%E7%A8%8B%E5%BA%8F%E5%91%98%E8%BF%9B%E4%BF%AE%E6%8C%87%E5%8D%97/"/>
<id>https://catinsides.github.io/2017/05/15/JS%E9%87%8E%E7%94%9F%E7%A8%8B%E5%BA%8F%E5%91%98%E8%BF%9B%E4%BF%AE%E6%8C%87%E5%8D%97/</id>
<published>2017-05-15T13:08:00.000Z</published>
<updated>2024-08-14T08:42:02.738Z</updated>
<content type="html"><![CDATA[<h3 id="基本"><a href="#基本" class="headerlink" title="基本"></a>基本</h3><p><code>缩进</code><br>4个空格，而不是一个制表符</p><p><code>结尾</code><br>分号结尾</p><p><code>行长度</code><br>80个字符</p><p><code>换行</code><br>符号结尾，第二行追加2单位(8字符)缩进</p><p><code>空行</code></p><ul><li>方法之间</li><li>方法中局部变量和第一条语句之间</li><li>注释之前</li><li>逻辑片段之间</li></ul><p><code>命名</code></p><ul><li>Camel Case</li><li>变量前缀为名词</li><li>函数前缀为动词</li><li>表数量 count, length, size</li><li>循环体 i, j, k</li><li>常量使用大写字母，下划线分隔单词</li><li>构造函数使用Pascal Case</li></ul><p><code>函数名参考</code></p><table><thead><tr><th>动词</th><th>含义</th></tr></thead><tbody><tr><td>can</td><td>返回布尔值</td></tr><tr><td>has</td><td>返回布尔值</td></tr><tr><td>is</td><td>返回布尔值</td></tr><tr><td>get</td><td>返回非布尔值</td></tr><tr><td>set</td><td>保存一个值</td></tr></tbody></table><p><code>直接量</code></p><ul><li>字符串 单引号、双引号皆可</li><li>数字，不使用八进制；不以小数点结尾</li></ul><p><code>null</code></p><ul><li>对象占位符</li><li>不要 检测是否传入某个参数</li><li>不要 检测一个未初始化的变量</li></ul><p><code>undefined</code><br>未被初始化的变量初始值</p><p><code>对象直接量</code></p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//不好的写法</span></span><br><span class="line"><span class="keyword">var</span> book = <span class="keyword">new</span> <span class="title class_">Object</span>();</span><br><span class="line">book.<span class="property">title</span> = <span class="string">"JavaScript"</span>;</span><br><span class="line">book.<span class="property">author</span> = <span class="string">"Mike"</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">//好的写法</span></span><br><span class="line"><span class="keyword">var</span> book = {</span><br><span class="line"> <span class="attr">title</span>: <span class="string">"JavaScript"</span>,</span><br><span class="line"> <span class="attr">author</span>: <span class="string">"Mike"</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><code>数组直接量</code></p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//不好的写法</span></span><br><span class="line"><span class="keyword">var</span> colors = <span class="keyword">new</span> <span class="title class_">Array</span>(<span class="string">"red"</span>, <span class="string">"blue"</span>, <span class="string">"green"</span>);</span><br><span class="line"><span class="keyword">var</span> numbers = <span class="keyword">new</span> <span class="title class_">Array</span>(<span class="number">1</span>, <span class="number">2</span>, <span class="number">3</span>);</span><br><span class="line"></span><br><span class="line"><span class="comment">//好的写法</span></span><br><span class="line"><span class="keyword">var</span> colors = [<span class="string">"red"</span>, <span class="string">"blue"</span>, <span class="string">"green"</span>];</span><br><span class="line"><span class="keyword">var</span> numbers = [<span class="number">1</span>, <span class="number">2</span>, <span class="number">3</span>];</span><br></pre></td></tr></table></figure><h3 id="注释"><a href="#注释" class="headerlink" title="注释"></a>注释</h3><p><code>单行注释</code><br>//something</p><ul><li>独占一行，解释下一行，注释前留一空行，缩进与下一行保持一致</li><li>尾部注释与代码保持一单位缩进，不应超过最大长度限制</li></ul><p><code>多行注释</code></p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">/*</span></span><br><span class="line"><span class="comment"> * 第一行注释</span></span><br><span class="line"><span class="comment"> * 第二行注释</span></span><br><span class="line"><span class="comment"> */</span></span><br></pre></td></tr></table></figure><p><code>注释时机</code></p><ul><li>难于理解的代码</li><li>可能被理解错误的代码</li><li>浏览器兼容性</li></ul><p><code>文档注释</code></p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment">这是一个文档注释</span></span><br><span class="line"><span class="comment">**/</span></span><br></pre></td></tr></table></figure><h3 id="语句和表达式"><a href="#语句和表达式" class="headerlink" title="语句和表达式"></a>语句和表达式</h3><p><code>基本</code></p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//好的写法</span></span><br><span class="line"><span class="keyword">if</span> (condition) {</span><br><span class="line"> <span class="title function_">doSomething</span>();</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">//不好的写法</span></span><br><span class="line"><span class="keyword">if</span> (condition)</span><br><span class="line"> <span class="title function_">doSomething</span>();</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> (condition) <span class="title function_">doSomething</span>();</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> (condition) { <span class="title function_">doSomething</span>(); }</span><br></pre></td></tr></table></figure><p>缩进应对齐<br>所有块语句都应使用花括号</p><ul><li>if</li><li>for</li><li>while</li><li>do…while…</li><li>try…catch…finally</li></ul><p><code>花括号对齐方式</code></p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//好的写法</span></span><br><span class="line"><span class="keyword">if</span> (condition) {</span><br><span class="line"> <span class="title function_">doSomething</span>();</span><br><span class="line">} <span class="keyword">else</span> {</span><br><span class="line"> <span class="title function_">doSomethingElse</span>();</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">//不好的写法</span></span><br><span class="line"><span class="keyword">if</span> (condition)</span><br><span class="line">{</span><br><span class="line"> <span class="title function_">doSomething</span>();</span><br><span class="line">}</span><br><span class="line"><span class="keyword">else</span></span><br><span class="line">{</span><br><span class="line"> <span class="title function_">doSomethingElse</span>();</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><code>块语句间隔</code></p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//好的写法</span></span><br><span class="line"><span class="keyword">if</span> (condition) {</span><br><span class="line"> <span class="title function_">doSomething</span>();</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="comment">//不好的写法</span></span><br><span class="line"><span class="keyword">if</span>(condition){</span><br><span class="line"> <span class="title function_">doSomething</span>();</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> ( condition ) {</span><br><span class="line"> <span class="title function_">doSomething</span>();</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><code>switch</code></p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">switch</span>(condition) {</span><br><span class="line"> <span class="keyword">case</span> <span class="string">"first"</span>:</span><br><span class="line"> <span class="comment">//代码</span></span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">case</span> <span class="string">"second"</span>:</span><br><span class="line"> <span class="comment">//代码</span></span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">case</span> <span class="string">"third"</span>:</span><br><span class="line"> <span class="comment">//代码</span></span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"></span><br><span class="line"> <span class="attr">default</span>:</span><br><span class="line"> <span class="comment">//代码</span></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><code>with</code><br>禁用</p><p><code>for</code><br>避免使用continue</p><p><code>for-in</code><br>不应遍历数组</p><h3 id="变量、函数和运算符"><a href="#变量、函数和运算符" class="headerlink" title="变量、函数和运算符"></a>变量、函数和运算符</h3><p><code>变量声明</code></p><ul><li>注意提升机制</li></ul><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//好的写法</span></span><br><span class="line"><span class="keyword">function</span> <span class="title function_">doSomethingWithItems</span>(<span class="params">items</span>) {</span><br><span class="line"> <span class="keyword">var</span> value = <span class="number">10</span>,</span><br><span class="line"> result = value + <span class="number">10</span>,</span><br><span class="line"> i,</span><br><span class="line"> len;</span><br><span class="line"></span><br><span class="line"> <span class="keyword">for</span> (i=<span class="number">0</span>, len=items.<span class="property">length</span>; i < len; i++) {</span><br><span class="line"> <span class="title function_">doSomething</span>(items[i]);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p><code>函数声明</code></p><ul><li>先定义后使用</li><li>不应在块语句内定义</li><li>函数名与左括号间无空格</li></ul><p><code>立即调用函数</code></p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//好的写法</span></span><br><span class="line"><span class="keyword">var</span> value = (<span class="keyword">function</span>(<span class="params"></span>) {</span><br><span class="line"> <span class="comment">//函数体</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> {</span><br><span class="line"> <span class="attr">message</span>: <span class="string">"Done"</span></span><br><span class="line"> }</span><br><span class="line">}());</span><br></pre></td></tr></table></figure><p><code>严格模式</code></p><ul><li>应在函数体内使用，而非全局</li></ul><p><code>相等</code></p><ul><li>应使用 ‘===’, ‘!==’, 而不是 ‘==’, ‘!=’</li></ul><p><code>eval()</code></p><ul><li>避免使用 eval(), Function<figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//不好的写法</span></span><br><span class="line"><span class="keyword">var</span> myfunc = <span class="keyword">new</span> <span class="title class_">Function</span>(<span class="string">"alert('Hi!')"</span>);</span><br><span class="line"></span><br><span class="line"><span class="built_in">setTimeout</span>(<span class="string">"document.body.style.background='red'"</span>, <span class="number">50</span>);</span><br><span class="line"></span><br><span class="line"><span class="built_in">setInterval</span>(<span class="string">"document.title = 'It is now'"</span> + (<span class="keyword">new</span> <span class="title class_">Date</span>()), <span class="number">1000</span>);</span><br><span class="line"></span><br><span class="line"><span class="comment">//而应使用匿名函数代替</span></span><br><span class="line"><span class="built_in">setTimeout</span>(<span class="keyword">function</span>(<span class="params"></span>) {</span><br><span class="line"> <span class="variable language_">document</span>.<span class="property">body</span>.<span class="property">style</span>.<span class="property">background</span>=<span class="string">'red'</span>;</span><br><span class="line">}, <span class="number">50</span>);</span><br></pre></td></tr></table></figure></li></ul><p><code>原始包装类型</code></p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//不好的写法</span></span><br><span class="line"><span class="keyword">var</span> name = <span class="keyword">new</span> <span class="title class_">String</span>(<span class="string">"Mike"</span>);</span><br><span class="line"><span class="keyword">var</span> author = <span class="keyword">new</span> <span class="title class_">Boolean</span>(ture);</span><br><span class="line"><span class="keyword">var</span> count = <span class="keyword">new</span> <span class="title class_">Number</span>(<span class="number">10</span>);</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">如何培养代码洁癖。</summary>
<category term="笔记" scheme="https://catinsides.github.io/tags/%E7%AC%94%E8%AE%B0/"/>
</entry>
<entry>
<title>码农札记</title>
<link href="https://catinsides.github.io/2017/05/15/%E7%A0%81%E5%86%9C%E6%9C%AD%E8%AE%B0/"/>
<id>https://catinsides.github.io/2017/05/15/%E7%A0%81%E5%86%9C%E6%9C%AD%E8%AE%B0/</id>
<published>2017-05-15T13:00:00.000Z</published>
<updated>2024-08-14T08:42:02.743Z</updated>
<content type="html"><![CDATA[<h3 id="Git"><a href="#Git" class="headerlink" title="Git"></a>Git</h3><h4 id="日常操作"><a href="#日常操作" class="headerlink" title="日常操作"></a>日常操作</h4><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 获取远端代码</span></span><br><span class="line">git fetch upstream xxxx</span><br><span class="line"></span><br><span class="line"><span class="comment"># 将远端代码合并到本地</span></span><br><span class="line">git merge upstream/xxxx</span><br><span class="line"></span><br><span class="line"><span class="comment"># 获取并合并代码</span></span><br><span class="line">git pull</span><br><span class="line"></span><br><span class="line"><span class="comment"># 查看不同(工作区与暂存区)</span></span><br><span class="line">git diff</span><br><span class="line"></span><br><span class="line"><span class="comment"># 查看工作区修改的文件</span></span><br><span class="line">git status</span><br><span class="line"></span><br><span class="line"><span class="comment"># 放弃工作区的修改</span></span><br><span class="line">git checkout</span><br><span class="line"></span><br><span class="line"><span class="comment"># 将修改的文件提交到暂存区</span></span><br><span class="line">git add</span><br><span class="line"></span><br><span class="line"><span class="comment"># 提交更改</span></span><br><span class="line">git commit -m <span class="string">"# 注释"</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 提交到远端</span></span><br><span class="line">git push origin xxxx</span><br><span class="line"></span><br><span class="line"><span class="comment"># 查看提交日志</span></span><br><span class="line">git <span class="built_in">log</span> (--pretty=oneline)</span><br><span class="line"></span><br><span class="line"><span class="comment"># 版本回退</span></span><br><span class="line">git reset --hard HEAD^ <span class="comment"># 上一个版本</span></span><br><span class="line">git reset --hard HEAD^^ <span class="comment"># 上上一个版本</span></span><br><span class="line">git reset --hard HEAD~20 <span class="comment"># 上20个版本</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 回到未来</span></span><br><span class="line">git reflog</span><br><span class="line">git reset</span><br><span class="line"></span><br><span class="line"><span class="comment"># 删除文件</span></span><br><span class="line">git <span class="built_in">rm</span></span><br><span class="line">git commit</span><br><span class="line"></span><br><span class="line"><span class="comment"># 合并一个commit</span></span><br><span class="line">git cherry-pick [commitId]</span><br><span class="line"></span><br><span class="line"><span class="comment"># 修改上一次commit注释</span></span><br><span class="line">git commit --amend -m <span class="string">'# 新的注释'</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 显示将要删除的文件</span></span><br><span class="line">git clean -n</span><br><span class="line"></span><br><span class="line"><span class="comment"># 删除没有track的文件，不包含.gitignore</span></span><br><span class="line">git clean -f</span><br><span class="line"></span><br><span class="line"><span class="comment"># 删除指定路径下没有track的文件</span></span><br><span class="line">git clean -f <path></span><br><span class="line"></span><br><span class="line"><span class="comment"># 删除当前目录下没有track的文件和文件夹</span></span><br><span class="line">git clean -fd</span><br><span class="line"></span><br><span class="line"><span class="comment"># 暂存更改</span></span><br><span class="line">git stash</span><br><span class="line"></span><br><span class="line"><span class="comment"># 弹出暂存</span></span><br><span class="line">git stash pop</span><br><span class="line"></span><br><span class="line"><span class="comment">#### 远程仓库</span></span><br><span class="line"><span class="comment"># 创建SSH KEY</span></span><br><span class="line"><span class="comment"># 用户目录 .ssh/</span></span><br><span class="line"><span class="comment"># id_rsa 私钥</span></span><br><span class="line"><span class="comment"># id_rsa.pub 公钥</span></span><br><span class="line">ssh-keygen -t rsa -C <span class="string">"[email protected]"</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 在Github上 Add SSH Key, 粘贴公钥内容</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 克隆远端代码</span></span><br><span class="line">git <span class="built_in">clone</span> some_url</span><br></pre></td></tr></table></figure><h4 id="进阶"><a href="#进阶" class="headerlink" title="进阶"></a>进阶</h4><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 创建并切换到分支</span></span><br><span class="line">git checkout -b dev</span><br><span class="line"></span><br><span class="line"><span class="comment"># 等同于</span></span><br><span class="line">git branch dev</span><br><span class="line">git checkout dev</span><br><span class="line"></span><br><span class="line"><span class="comment"># 查看分支</span></span><br><span class="line">git branch</span><br><span class="line"></span><br><span class="line"><span class="comment"># 合并分支</span></span><br><span class="line">git merge</span><br><span class="line"></span><br><span class="line"><span class="comment"># 删除分支</span></span><br><span class="line">git branch -d</span><br><span class="line"></span><br><span class="line"><span class="comment"># 查看分支合并图</span></span><br><span class="line">git <span class="built_in">log</span> --graph</span><br><span class="line"></span><br><span class="line"><span class="comment"># 分支策略</span></span><br><span class="line">git merge --no-ff -m <span class="string">"注释"</span> dev</span><br></pre></td></tr></table></figure><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># Bug分支</span></span><br><span class="line"><span class="comment"># 暂存修改</span></span><br><span class="line">git stash</span><br><span class="line"></span><br><span class="line"><span class="comment"># 回到master，并创建新分支，修复后提交</span></span><br><span class="line">git checkout master</span><br><span class="line">git checkout -b issue-101</span><br><span class="line">git commit -m <span class="string">"fix-issue-101"</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 合并bug分支并删除</span></span><br><span class="line">git checkout master</span><br><span class="line">git merge --no-ff -m <span class="string">"m-merge-issue-101"</span> issue-101</span><br><span class="line">git branch -d issue-101</span><br><span class="line"></span><br><span class="line"><span class="comment"># 将修复bug后的master合并到dev</span></span><br><span class="line">git checkout dev</span><br><span class="line">git merge --no-ff -m <span class="string">"dev-merge-m"</span> master</span><br><span class="line"></span><br><span class="line"><span class="comment"># 弹出暂存的修改</span></span><br><span class="line">git stash pop</span><br><span class="line"></span><br><span class="line"><span class="comment"># 修复冲突</span></span><br><span class="line">git commit -m <span class="string">"fixconflict & append something"</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 完成后提交</span></span><br><span class="line">git checkout master</span><br><span class="line">git merge --no-ff -m <span class="string">"m-merge-dev"</span> dev</span><br><span class="line">git branch -d dev</span><br></pre></td></tr></table></figure><h4 id="标签管理"><a href="#标签管理" class="headerlink" title="标签管理"></a>标签管理</h4><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 创建标签，默认HEAD，也可指定commit id</span></span><br><span class="line">git tag <name></span><br><span class="line"></span><br><span class="line"><span class="comment"># 指定标签信息</span></span><br><span class="line">git tag -a <tagname> -m <span class="string">"message"</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># PGP签名标签</span></span><br><span class="line">git tag -s <tagname> -m <span class="string">"message"</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 查看所有标签</span></span><br><span class="line">git tag</span><br><span class="line"></span><br><span class="line"><span class="comment"># 推送本地标签</span></span><br><span class="line">git push origin <tagname></span><br><span class="line"></span><br><span class="line"><span class="comment"># 推送全部本地未推送的标签</span></span><br><span class="line">git push origin --tags</span><br><span class="line"></span><br><span class="line"><span class="comment"># 删除一个本地标签</span></span><br><span class="line">git tag -d <tagname></span><br><span class="line"></span><br><span class="line"><span class="comment"># 删除一个远程标签</span></span><br><span class="line">git push origin :refs/tags/<tagname></span><br></pre></td></tr></table></figure><h3 id="Linux"><a href="#Linux" class="headerlink" title="Linux"></a>Linux</h3><h4 id="文件命令"><a href="#文件命令" class="headerlink" title="文件命令"></a>文件命令</h4><table><thead><tr><th>命令</th><th>操作</th></tr></thead><tbody><tr><td>ls</td><td>列出目录</td></tr><tr><td>ls -al</td><td>使用格式化列出隐藏文件</td></tr><tr><td>cd dir</td><td>更改目录到dir</td></tr><tr><td>cd</td><td>更改到home目录</td></tr><tr><td>pwd</td><td>显示当前目录</td></tr><tr><td>mkdir dir</td><td>创建目录dir</td></tr><tr><td>rm file</td><td>删除file</td></tr><tr><td>rm -r dir</td><td>删除目录dir</td></tr><tr><td>rm -f file</td><td>强制删除file</td></tr><tr><td>rm -rf dir</td><td>强制删除目录dir</td></tr><tr><td>cp file1 file2</td><td>将file1复制到file2</td></tr><tr><td>cp -r dir1 dir2</td><td>将file1复制到dir2，如果dir2不存在则创建它</td></tr><tr><td>mv file1 file2</td><td>将file1重命名或移动到file2，若file2存在则移动</td></tr><tr><td>ln -s file link</td><td>创建file的符号连接link</td></tr><tr><td>touch file</td><td>创建file</td></tr><tr><td>cat > file</td><td>将标准输入添加到file</td></tr><tr><td>more file</td><td>查看file的内容</td></tr><tr><td>head file</td><td>查看file的前10行</td></tr><tr><td>tail file</td><td>查看file的后10行</td></tr><tr><td>tail -f file</td><td>从后10行开始查看file的内容</td></tr></tbody></table><h4 id="进程管理"><a href="#进程管理" class="headerlink" title="进程管理"></a>进程管理</h4><table><thead><tr><th>命令</th><th>操作</th></tr></thead><tbody><tr><td>ps</td><td>显示当前的活动进程</td></tr><tr><td>top</td><td>显示所有正在运行的进程</td></tr><tr><td>kill pid</td><td>杀掉进程id pid</td></tr><tr><td>killall proc</td><td>杀掉所有名为proc的进程</td></tr><tr><td>bg</td><td>列出已停止或后台的作业</td></tr><tr><td>fg</td><td>将最近的作业带到前台</td></tr><tr><td>fg n</td><td>将作业n带到前台</td></tr></tbody></table><h4 id="文件权限"><a href="#文件权限" class="headerlink" title="文件权限"></a>文件权限</h4><table><thead><tr><th>命令</th><th>操作</th></tr></thead><tbody><tr><td>chmod octal file</td><td>更改file的权限</td></tr></tbody></table><ul><li>4 读（r）</li><li>2 写（w）</li><li>1 执行（x）</li></ul><h4 id="SSH"><a href="#SSH" class="headerlink" title="SSH"></a>SSH</h4><table><thead><tr><th>命令</th><th>操作</th></tr></thead><tbody><tr><td>ssh user@host</td><td>以user用户连接到host</td></tr><tr><td>ssh -p port user@host</td><td>在端口port以user用户身份连接到host</td></tr><tr><td>ssh-copy-id user@host</td><td>将密钥添加到host以实现无密码登录</td></tr></tbody></table><h4 id="搜索"><a href="#搜索" class="headerlink" title="搜索"></a>搜索</h4><table><thead><tr><th>命令</th><th>操作</th></tr></thead><tbody><tr><td>grep pattern files</td><td>搜索files中匹配pattern的内容</td></tr><tr><td>grep -r pattern dir</td><td>递归搜索dir中匹配pattern的内容</td></tr><tr><td>command l grep pattern</td><td>搜索command输出中匹配pattern的内容</td></tr></tbody></table><h4 id="系统信息"><a href="#系统信息" class="headerlink" title="系统信息"></a>系统信息</h4><table><thead><tr><th>命令</th><th>操作</th></tr></thead><tbody><tr><td>date</td><td>显示当前日期和时间</td></tr><tr><td>cal</td><td>显示当月的日历</td></tr><tr><td>uptime</td><td>显示系统从开机到现在所运行的时间</td></tr><tr><td>w</td><td>显示登录的用户</td></tr><tr><td>whoami</td><td>查看你的当前用户名</td></tr><tr><td>finger user</td><td>显示user的相关信息</td></tr><tr><td>uname -a</td><td>显示内核信息</td></tr><tr><td>cat /proc/cpuinfo</td><td>查看cpu的信息</td></tr><tr><td>cat /proc/meminfo</td><td>查看内存信息</td></tr><tr><td>man command</td><td>显示command的说明手册</td></tr><tr><td>df</td><td>显示磁盘占用情况</td></tr><tr><td>du</td><td>显示目录空间占用情况</td></tr><tr><td>free</td><td>显示内存及交换区占用情况</td></tr></tbody></table><h4 id="压缩"><a href="#压缩" class="headerlink" title="压缩"></a>压缩</h4><table><thead><tr><th>命令</th><th>操作</th></tr></thead><tbody><tr><td>tar cf file.tar files</td><td>创建包含files的tar文件file.tar</td></tr><tr><td>tar xf file.tar</td><td>从file.tar提取文件</td></tr><tr><td>tar czf file.tar.gz files</td><td>使用Gzip压缩创建tar文件</td></tr><tr><td>tar xzf file.tar.gz</td><td>使用Gzip提取tar文件</td></tr><tr><td>tar cjf file.tar.bz2</td><td>使用Bzip2压缩创建tar文件</td></tr><tr><td>tar xjf file.tar.bz2</td><td>使用Bzip2提取tar文件</td></tr><tr><td>gzip file</td><td>压缩file并重命名为file.gz</td></tr><tr><td>gzip -d file.gz</td><td>将file.gz解压缩为file</td></tr></tbody></table><h4 id="网络"><a href="#网络" class="headerlink" title="网络"></a>网络</h4><table><thead><tr><th>命令</th><th>操作</th></tr></thead><tbody><tr><td>ping host</td><td>ping host并输出结果</td></tr><tr><td>whois domain</td><td>获取domain的whois信息</td></tr><tr><td>dig domain</td><td>获取domain的DNS信息</td></tr><tr><td>dig -x host</td><td>逆向查询host</td></tr><tr><td>wget file</td><td>下载file</td></tr><tr><td>wget -c file</td><td>断点续传</td></tr></tbody></table><h4 id="安装"><a href="#安装" class="headerlink" title="安装"></a>安装</h4><p>从源代码安装：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">./configure</span><br><span class="line">make</span><br><span class="line">make install</span><br><span class="line">dpkg -i pkg.deb</span><br><span class="line">rpm -Uvh pkg.rpm</span><br></pre></td></tr></table></figure><h4 id="快捷键"><a href="#快捷键" class="headerlink" title="快捷键"></a>快捷键</h4><table><thead><tr><th>命令</th><th>操作</th></tr></thead><tbody><tr><td>Ctrl + C</td><td>停止当前命令</td></tr><tr><td>Ctrl + Z</td><td>停止当前命令，并使用fg恢复</td></tr><tr><td>Ctrl + D</td><td>注销当前会话，与exit相似</td></tr><tr><td>Ctrl + W</td><td>删除当前行中的字</td></tr><tr><td>Ctrl + U</td><td>删除整行</td></tr><tr><td>!!</td><td>重复上次的命令</td></tr><tr><td>exit</td><td>注销当前会话</td></tr></tbody></table>]]></content>
<summary type="html">记录常用Linux操作.</summary>
<category term="笔记" scheme="https://catinsides.github.io/tags/%E7%AC%94%E8%AE%B0/"/>
</entry>
<entry>
<title>Composer搭建MVC框架</title>
<link href="https://catinsides.github.io/2017/03/30/Composer%E6%90%AD%E5%BB%BAMVC%E6%A1%86%E6%9E%B6/"/>
<id>https://catinsides.github.io/2017/03/30/Composer%E6%90%AD%E5%BB%BAMVC%E6%A1%86%E6%9E%B6/</id>
<published>2017-03-30T03:11:28.000Z</published>
<updated>2024-08-14T08:42:02.738Z</updated>
<content type="html"><![CDATA[<blockquote><p>Composer是一个依赖管理工具，利用PSR标准和PHP命名空间的特性，构造出繁荣的PHP生态系统。</p></blockquote><h3 id="Composer是什么？"><a href="#Composer是什么？" class="headerlink" title="Composer是什么？"></a>Composer是什么？</h3><p><strong>先从别处摘抄一段简介，感受一下Composer的功能。</strong><br>Composer不是一个包管理器。它涉及”pachages”和”libraries”，但它在每个项目的基础上进行管理，在你项目的某个目录中（例如vendor）进行安装。默认情况下它不会在全局安装任何东西。因此，这仅仅是一个依赖管理。</p><p>这种想法并不新鲜，Composer受到了node’s npm和ruby’s bundler的强烈启发。而当时PHP下并没有类似的工具。</p><p>Composer将这样为你解决问题：</p><p>1.你有一个项目依赖于若干个库。<br>2.其中一些库依赖于其他库。<br>3.你声明你所依赖的东西。<br>4.Composer会找出哪个版本的包需要安装，并安装它们（将他们下载到你的项目中）。</p><p>好了，了解功能之后，lets coding！</p><h3 id="准备工作"><a href="#准备工作" class="headerlink" title="准备工作"></a>准备工作</h3><h4 id="Composer安装"><a href="#Composer安装" class="headerlink" title="Composer安装"></a>Composer安装</h4><p><a href="http://docs.phpcomposer.com/00-intro.html#Installation-Windows">百度一下</a></p><h4 id="建立项目"><a href="#建立项目" class="headerlink" title="建立项目"></a>建立项目</h4><p>在一个目录下建立文件夹，名字就叫smvc(simple mvc)吧。<br>在文件夹下新建文件composer.json:</p><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="punctuation">{</span></span><br><span class="line"><span class="attr">"require"</span><span class="punctuation">:</span><span class="punctuation">{</span></span><br><span class="line"></span><br><span class="line"><span class="punctuation">}</span></span><br><span class="line"><span class="punctuation">}</span></span><br></pre></td></tr></table></figure><p>在此目录下，开启命令行运行：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">composer update</span><br></pre></td></tr></table></figure><p>几秒钟后会生成以下文件：</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">|--vendor/</span><br><span class="line">||--scrapy.cfg</span><br><span class="line">|||--autoload_classmap.php</span><br><span class="line">|||--autoload_namespaces.php</span><br><span class="line">|||--autoload_psr4.php</span><br><span class="line">|||--autoload_real.php</span><br><span class="line">|||--autoload_static.php</span><br><span class="line">|||--ClassLoader.php</span><br><span class="line">|||--installed.json</span><br><span class="line">|||--LICENSE</span><br><span class="line">||--autoload.php</span><br><span class="line">|--composer.json</span><br></pre></td></tr></table></figure><p>OK,准备工作完成，可以开工了。</p><h3 id="MVC搭建"><a href="#MVC搭建" class="headerlink" title="MVC搭建"></a>MVC搭建</h3><p>在上一篇文章中<a href="https://catinsides.github.io/catinsides.github.io/2017/01/26/%E5%A6%82%E4%BD%95%E7%BC%96%E5%86%99%E8%87%AA%E5%B7%B1%E7%9A%84MVC%E6%A1%86%E6%9E%B6/">如何编写自己的MVC框架</a>已经简单介绍了MVC框架的结构和运作流程。所以不多废话，直接按照MVC的思路逐步搭建出一个小型的MVC框架。</p><h4 id="路由"><a href="#路由" class="headerlink" title="路由"></a>路由</h4><p>既然用了Composer，就不需要像之前一样自己手动撸代码了，而是直接到全球最大同性交友平台Github搜索关键字router，<a href="https://github.com/search?l=PHP&q=router&type=Repositories&utf8=%E2%9C%93">搜一下</a><br>选择一个契(xi)合(huan)项目的包，使用composer进行下载。这里我选择的是klein/klein.php(a fast&flexible router)。没有什么特殊的理由，只是因为出现在了首页，并且stars数量最多而已。<br>然后打开composer.json，添加安装信息（一般情况下会列在开发者文档中）:</p><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="punctuation">{</span></span><br><span class="line"><span class="attr">"require"</span><span class="punctuation">:</span><span class="punctuation">{</span></span><br><span class="line"><span class="attr">"klein/klein"</span><span class="punctuation">:</span> <span class="string">"dev-master"</span></span><br><span class="line"><span class="punctuation">}</span></span><br><span class="line"><span class="punctuation">}</span></span><br></pre></td></tr></table></figure><p>然后再命令行中运行<code>composer update</code>,等待安装下载完成，速度视网络情况而定。<br>安装完成后，会在<code>vender</code>文件夹生成相关目录。<br>在根目录下建立入口文件<code>index.php</code>：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"><?php</span></span><br><span class="line"><span class="comment">//自动加载</span></span><br><span class="line"><span class="keyword">require_once</span> <span class="string">'./vendor/autoload.php'</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">//载入路由文件</span></span><br><span class="line"><span class="keyword">require</span> <span class="string">'./core/routes.php'</span>;</span><br></pre></td></tr></table></figure><p>然后新建文件夹core，在里面新建routes.php，使用刚才下载的路由包:</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"><?php</span></span><br><span class="line"><span class="variable">$klein</span> = <span class="keyword">new</span> <span class="title class_">\Klein\Klein</span>();</span><br><span class="line"></span><br><span class="line"><span class="variable">$klein</span>-><span class="title function_ invoke__">respond</span>(<span class="string">'GET'</span>, <span class="string">'/hello'</span>, function () {</span><br><span class="line"> <span class="keyword">return</span> <span class="string">"hello world!"</span>;</span><br><span class="line">});</span><br><span class="line"></span><br><span class="line"><span class="variable">$klein</span>-><span class="title function_ invoke__">dispatch</span>();</span><br></pre></td></tr></table></figure><p>打开wamp在本地测试一下能否成功运行。点击Localhost后自动打开浏览器，在地址栏地址后输入<code>/hello</code>会出现hello world！字样，OK运行成功。其实每个包的作者都会在项目中写出详细的开发文档，只要按照给定的API设定好路由即可。在这里以能够实现数据库常用的增删改查为目标构建出这个小型框架。</p><p>根据逻辑重构路由文件：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"><?php</span></span><br><span class="line"><span class="variable">$klein</span> = <span class="keyword">new</span> <span class="title class_">\Klein\Klein</span>();</span><br><span class="line"></span><br><span class="line"><span class="comment">//设定默认主页，各项请求在这里完成</span></span><br><span class="line"><span class="variable">$klein</span>-><span class="title function_ invoke__">respond</span>(function () {</span><br><span class="line"> <span class="keyword">include</span> <span class="string">'./tpl/index.php'</span>;</span><br><span class="line">});</span><br><span class="line"></span><br><span class="line"><span class="comment">//将路由指向控制器文件</span></span><br><span class="line"><span class="variable">$klein</span>-><span class="title function_ invoke__">respond</span>(<span class="string">'GET'</span>, <span class="string">'/create'</span>, function () {</span><br><span class="line"> <span class="title class_">Controller</span>::<span class="title function_ invoke__">create</span>();</span><br><span class="line">});</span><br><span class="line"></span><br><span class="line"><span class="variable">$klein</span>-><span class="title function_ invoke__">respond</span>(<span class="string">'GET'</span>, <span class="string">'/update'</span>, function () {</span><br><span class="line"> <span class="title class_">Controller</span>::<span class="title function_ invoke__">update</span>();</span><br><span class="line">});</span><br><span class="line"></span><br><span class="line"><span class="variable">$klein</span>-><span class="title function_ invoke__">respond</span>(<span class="string">'GET'</span>, <span class="string">'/retrieve'</span>, function () {</span><br><span class="line"> <span class="title class_">Controller</span>::<span class="title function_ invoke__">retrieve</span>();</span><br><span class="line">});</span><br><span class="line"></span><br><span class="line"><span class="variable">$klein</span>-><span class="title function_ invoke__">respond</span>(<span class="string">'GET'</span>, <span class="string">'/delete'</span>, function () {</span><br><span class="line"> <span class="title class_">Controller</span>::<span class="title function_ invoke__">delete</span>();</span><br><span class="line">});</span><br><span class="line"></span><br><span class="line"><span class="variable">$klein</span>-><span class="title function_ invoke__">dispatch</span>();</span><br></pre></td></tr></table></figure><h4 id="模型"><a href="#模型" class="headerlink" title="模型"></a>模型</h4><p>在根目录创建文件夹config，新建config.php文件写入数据库配置信息：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"><?php</span></span><br><span class="line"><span class="title function_ invoke__">define</span>(<span class="string">"DB_HOST"</span>, <span class="string">"localhost"</span>);</span><br><span class="line"><span class="title function_ invoke__">define</span>(<span class="string">"DB_USER"</span>, <span class="string">"root"</span>);</span><br><span class="line"><span class="title function_ invoke__">define</span>(<span class="string">"DB_PASSWORD"</span>, <span class="string">"1234"</span>);</span><br><span class="line"><span class="title function_ invoke__">define</span>(<span class="string">"DB_NAME"</span>, <span class="string">"test"</span>);</span><br></pre></td></tr></table></figure><p>在core文件夹创建基本的模型类Model.php,在里面使用PDO编写基本的数据库操作。</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"><?php</span> </span><br><span class="line"><span class="comment">//引入数据库配置文件</span></span><br><span class="line"><span class="keyword">require</span> <span class="string">'./config/config.php'</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Model</span></span></span><br><span class="line"><span class="class"></span>{</span><br><span class="line"><span class="keyword">protected</span> <span class="variable">$_dbHandle</span>;</span><br><span class="line"><span class="keyword">protected</span> <span class="variable">$_table</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">__construct</span>(<span class="params"><span class="variable">$table</span></span>)</span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"><span class="variable language_">$this</span> -><span class="title function_ invoke__">connect</span>(DB_HOST, DB_USER, DB_PASSWORD, DB_NAME);</span><br><span class="line"><span class="variable language_">$this</span> ->_table = <span class="variable">$table</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">connect</span>(<span class="params"><span class="variable">$host</span>, <span class="variable">$user</span>, <span class="variable">$pass</span>, <span class="variable">$dbname</span></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="variable">$dsn</span> = <span class="title function_ invoke__">sprintf</span>(<span class="string">"mysql:host=%s;dbname=%s;charset=utf8"</span>, <span class="variable">$host</span>, <span class="variable">$dbname</span>);</span><br><span class="line"> <span class="variable">$option</span> = <span class="keyword">array</span>(PDO::<span class="variable constant_">ATTR_DEFAULT_FETCH_MODE</span> => PDO::<span class="variable constant_">FETCH_ASSOC</span>);</span><br><span class="line"> <span class="variable language_">$this</span>->_dbHandle = <span class="keyword">new</span> <span class="title function_ invoke__">PDO</span>(<span class="variable">$dsn</span>, <span class="variable">$user</span>, <span class="variable">$pass</span>, <span class="variable">$option</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (PDOException <span class="variable">$e</span>) {</span><br><span class="line"> <span class="keyword">exit</span>(<span class="string">'Wrong: '</span> . <span class="variable">$e</span>-><span class="title function_ invoke__">getMessage</span>());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">select</span>(<span class="params"></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="variable">$sql</span> = <span class="title function_ invoke__">sprintf</span>(<span class="string">"SELECT * FROM %s"</span>, <span class="variable">$this</span>->_table);</span><br><span class="line"> <span class="variable">$sth</span> = <span class="variable language_">$this</span>->_dbHandle-><span class="title function_ invoke__">prepare</span>(<span class="variable">$sql</span>);</span><br><span class="line"> <span class="variable">$sth</span>-><span class="title function_ invoke__">execute</span>();</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> <span class="variable">$sth</span>-><span class="title function_ invoke__">fetch</span>();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">add</span>(<span class="params"><span class="variable">$id</span>, <span class="variable">$data</span></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="variable">$sql</span> = <span class="title function_ invoke__">sprintf</span>(<span class="string">"INSERT INTO %s(id, name) VALUES (%d, '%s')"</span>, <span class="variable">$this</span>->_table, <span class="variable">$id</span>, <span class="variable">$data</span>);</span><br><span class="line"> <span class="variable language_">$this</span> ->_dbHandle-><span class="title function_ invoke__">query</span>(<span class="variable">$sql</span>);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">delete</span>(<span class="params"><span class="variable">$id</span></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="variable">$sql</span> = <span class="title function_ invoke__">sprintf</span>(<span class="string">"DELETE FROM %s WHERE `id` = %d"</span>, <span class="variable">$this</span>->_table, <span class="variable">$id</span>);</span><br><span class="line"> <span class="variable">$sth</span> = <span class="variable language_">$this</span>->_dbHandle-><span class="title function_ invoke__">prepare</span>(<span class="variable">$sql</span>);</span><br><span class="line"> <span class="variable">$sth</span>-><span class="title function_ invoke__">execute</span>();</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">update</span>(<span class="params"><span class="variable">$id</span>, <span class="variable">$data</span></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="variable">$sql</span> = <span class="title function_ invoke__">sprintf</span>(<span class="string">"UPDATE %s SET name = '%s' WHERE `id` = %d"</span>, <span class="variable">$this</span>->_table, <span class="variable">$data</span>, <span class="variable">$id</span>);</span><br><span class="line"> <span class="variable language_">$this</span> ->_dbHandle-><span class="title function_ invoke__">query</span>(<span class="variable">$sql</span>);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>数据库文件编写完成。</p><h4 id="控制器"><a href="#控制器" class="headerlink" title="控制器"></a>控制器</h4><p>在core文件夹新建控制器类Controller.php,在其内部使用Model.php与数据库进行连接:</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"><?php</span> </span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Controller</span></span>{</span><br><span class="line"><span class="keyword">public</span> <span class="built_in">static</span> <span class="function"><span class="keyword">function</span> <span class="title">create</span>(<span class="params"></span>)</span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"><span class="variable">$model</span> = <span class="keyword">new</span> <span class="title class_">Model</span>(<span class="string">'test'</span>);</span><br><span class="line"><span class="variable">$model</span> -><span class="title function_ invoke__">add</span>(<span class="string">'1'</span>, <span class="string">'Mike'</span>);</span><br><span class="line"><span class="keyword">echo</span> <span class="string">"插入成功！"</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="built_in">static</span> <span class="function"><span class="keyword">function</span> <span class="title">update</span>(<span class="params"></span>)</span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"><span class="variable">$model</span> = <span class="keyword">new</span> <span class="title class_">Model</span>(<span class="string">'test'</span>);</span><br><span class="line"><span class="variable">$model</span> -><span class="title function_ invoke__">update</span>(<span class="string">'1'</span>, <span class="string">'John'</span>);</span><br><span class="line"><span class="keyword">echo</span> <span class="string">"更新成功！"</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="built_in">static</span> <span class="function"><span class="keyword">function</span> <span class="title">retrieve</span>(<span class="params"></span>)</span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"><span class="variable">$model</span> = <span class="keyword">new</span> <span class="title class_">Model</span>(<span class="string">'test'</span>);</span><br><span class="line"><span class="variable">$results</span> = <span class="variable">$model</span> -><span class="title function_ invoke__">select</span>();</span><br><span class="line"><span class="title function_ invoke__">extract</span>(<span class="variable">$results</span>);</span><br><span class="line"><span class="keyword">require</span> <span class="string">"./tpl/index.php"</span>;</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="keyword">public</span> <span class="built_in">static</span> <span class="function"><span class="keyword">function</span> <span class="title">delete</span>(<span class="params"></span>)</span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line"><span class="variable">$model</span> = <span class="keyword">new</span> <span class="title class_">Model</span>(<span class="string">'test'</span>);</span><br><span class="line"><span class="variable">$model</span> -><span class="title function_ invoke__">delete</span>(<span class="string">'1'</span>);</span><br><span class="line"><span class="keyword">echo</span> <span class="string">"删除成功！"</span>;</span><br><span class="line">}</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h4 id="数据库"><a href="#数据库" class="headerlink" title="数据库"></a>数据库</h4><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> `test`(</span><br><span class="line">`id` <span class="type">int</span>(<span class="number">1</span>),</span><br><span class="line">`name` <span class="type">varchar</span>(<span class="number">5</span>)</span><br><span class="line">)ENGINE<span class="operator">=</span>Innodb <span class="keyword">DEFAULT</span> CHARSET<span class="operator">=</span>utf8;</span><br><span class="line"></span><br></pre></td></tr></table></figure><h4 id="自动加载"><a href="#自动加载" class="headerlink" title="自动加载"></a>自动加载</h4><p>类文件编写完成后，如果在首页测试会出现找不到相关类的错误。因为还没有在composer.json中设置为自动加载，所以在文件中添加字段，最终代码为：</p><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="punctuation">{</span></span><br><span class="line"><span class="attr">"require"</span><span class="punctuation">:</span><span class="punctuation">{</span></span><br><span class="line"><span class="attr">"klein/klein"</span><span class="punctuation">:</span> <span class="string">"dev-master"</span></span><br><span class="line"><span class="punctuation">}</span><span class="punctuation">,</span></span><br><span class="line"><span class="attr">"autoload"</span><span class="punctuation">:</span><span class="punctuation">{</span></span><br><span class="line"><span class="attr">"classmap"</span><span class="punctuation">:</span><span class="punctuation">[</span></span><br><span class="line"><span class="string">"core"</span></span><br><span class="line"><span class="punctuation">]</span></span><br><span class="line"><span class="punctuation">}</span></span><br><span class="line"><span class="punctuation">}</span></span><br><span class="line"></span><br></pre></td></tr></table></figure><p>这个MVC框架的视图层没有特别复杂的实现，而是直接采用了加载文件，输出变量的方式。<br>至此，这个框架的基本要求已经全部实现了。</p><h3 id="小结"><a href="#小结" class="headerlink" title="小结"></a>小结</h3><p>在本文中，简单利用了composer的下载和自动加载类的功能，实现了MVC框架。虽然这个框架瘦骨嶙峋到几乎只有骨骼……代码还有非常大的优化和拓展空间。而这些就可以用composer强大的依赖管理来实现。比如项目中需要一个验证码的功能，只需要下载并注册到composer.json中，然后正确引入即可使用。Composer这些功能特性极大方便了PHP开发者的快速开发。</p>]]></content>
<summary type="html">Composer是一个依赖管理工具，利用PSR标准和PHP命名空间的特性，构造出繁荣的PHP生态系统。</summary>
<category term="PHP" scheme="https://catinsides.github.io/tags/PHP/"/>
</entry>
<entry>
<title>SCRAPY爬取智联招聘信息</title>
<link href="https://catinsides.github.io/2017/02/07/SCRAPY%E7%88%AC%E5%8F%96%E6%99%BA%E8%81%94%E6%8B%9B%E8%81%98%E4%BF%A1%E6%81%AF/"/>
<id>https://catinsides.github.io/2017/02/07/SCRAPY%E7%88%AC%E5%8F%96%E6%99%BA%E8%81%94%E6%8B%9B%E8%81%98%E4%BF%A1%E6%81%AF/</id>
<published>2017-02-07T05:41:00.000Z</published>
<updated>2024-08-14T08:42:02.741Z</updated>
<content type="html"><![CDATA[<p>网上找工作这件事，首先要登录主站搜索职位名称，然后在一大串列表里逐个查看职位需求，查看完毕关闭页面，再打开新页面，想一想都觉得是费心费力的一件事。俗话说：“磨刀不误砍柴工”，再加上自己懒癌发作，心想着一定要一劳永逸的解决这件事。正好SCRAPY框架能派上用场，大致整理了一下思路开始撸代码。</p><h3 id="思路"><a href="#思路" class="headerlink" title="思路"></a>思路</h3><p>根据文章开头部分的话，可以整理出<code>网上找工作</code>这一系列动作的流程</p><ol><li>打开主站搜索<code>职位</code>，并选择<code>地点</code></li><li>网站返回一系列<code><li><a>职位名称</a></li></code></li><li>点击<code><a></code>进入详情页，里面包含着职位和公司的详细信息</li><li>查看完毕后关闭，换下一个<code><a></code></li></ol><p>好了，流程整理好了，怎么实现呢？<br>显然第一步打开网站获取网页信息可以使用<code>urllib</code>或者<code>requests</code>模块，第二步提取<code>URL</code>信息交给<code>re</code>和<code>BeautifulSoup</code>处理，第三步使用<code>SCRAPY</code>处理第二步获取的<code>URL</code>到逐个页面提取职位详细信息，使用<code>SCRAPY</code>内置的<code>xpath</code>即可。<br>思路整理完毕，let’s coding!</p><h3 id="获取URL"><a href="#获取URL" class="headerlink" title="获取URL"></a>获取URL</h3><p>秉着模块化的精神，我把提取URL这一部分单独写一个文件，然后在<code>SCRAPY</code>调用。这样不仅方便调试，还能减少<code>SCRAPY</code>内爬虫文件代码的冗杂程度。<br>首先，引入必要的模块</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># -*-coding:utf-8-*-</span></span><br><span class="line"><span class="keyword">import</span> urllib2</span><br><span class="line"><span class="keyword">import</span> re</span><br><span class="line"><span class="keyword">from</span> bs4 <span class="keyword">import</span> BeautifulSoup</span><br><span class="line"></span><br></pre></td></tr></table></figure><p>为了在其他模组内调用，创建一个<code>Geturls</code>类</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">Geturls</span>():</span><br></pre></td></tr></table></figure><p>其他方法都在这里完成。<br>定义打开网页的方法</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">open</span>(<span class="params">self, url</span>):</span><br><span class="line"><span class="keyword">try</span>:</span><br><span class="line">request = urllib2.Request(url)</span><br><span class="line">response = urllib2.urlopen(request)</span><br><span class="line">content = response.read().decode(<span class="string">'utf-8'</span>)</span><br><span class="line"><span class="keyword">return</span> content</span><br><span class="line"><span class="keyword">except</span> urllib2.URLError, e:</span><br><span class="line"><span class="keyword">if</span> <span class="built_in">hasattr</span>(e,<span class="string">"code"</span>):</span><br><span class="line"><span class="built_in">print</span> e.code</span><br><span class="line"><span class="keyword">if</span> <span class="built_in">hasattr</span>(e,<span class="string">"reason"</span>):</span><br><span class="line"><span class="built_in">print</span> e.reason</span><br></pre></td></tr></table></figure><p>打开网页成功后将返回网页内所有的内容，然后提取关键信息</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">distill</span>(<span class="params">self, content</span>):</span><br><span class="line">soup = BeautifulSoup(content, <span class="string">'html.parser'</span>).find_all(href=re.<span class="built_in">compile</span>(<span class="string">"http://jobs.zhaopin.com/"</span>))</span><br><span class="line">urls_temp = soup[:<span class="number">60</span>]</span><br><span class="line">urls = []</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> url <span class="keyword">in</span> urls_temp:</span><br><span class="line">pattern = re.<span class="built_in">compile</span>(<span class="string">'href="(.*?)"'</span>, re.S)</span><br><span class="line">url_temp = re.findall(pattern, <span class="built_in">str</span>(url))</span><br><span class="line">urls.append(url_temp)</span><br><span class="line"><span class="keyword">return</span> urls</span><br></pre></td></tr></table></figure><p>原本想把方法名设置为<code>extract</code>，但是<code>SCRAPY</code>内有相同的方法名，只好换成<code>distill</code>(蒸馏)，我觉得反而更加形象XD.<br>智联网页上的东西很多，而我只需要职位的链接，所以使用正则把链接提出来即可。这些链接的规律就是开头皆为<code>http://jobs.zhaopin.com/</code>,后面附带一系列数字。查看网页源代码后，发现还是蛮有规律的，在第一链接之前没有其他的广告链接，而且每页的职位信息顺序排列只有60个，在这之后又会有11个广告链接，如下图</p><p><img src="http://okzvvfkr0.bkt.clouddn.com/zhilian_urls.jpg"></p><p>所以只需要提取出前60个即可。</p><h3 id="获取信息"><a href="#获取信息" class="headerlink" title="获取信息"></a>获取信息</h3><h4 id="准备工作"><a href="#准备工作" class="headerlink" title="准备工作"></a>准备工作</h4><p>接下来的工作是，在上一步获得了所有的url之后，传到<code>SCRAPY</code>逐个打开页面获取职位信息。在目标目录内打开命令行执行</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">scrapy startproject recruit</span><br></pre></td></tr></table></figure><p>好吧，这个项目名字不是很高大上，起码能够望文生义吧……<br>命令执行后，会自动生成下列文件夹</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">|--recruit/</span><br><span class="line">||--scrapy.cfg</span><br><span class="line">||--recruit/</span><br><span class="line">||--`__init__.py`</span><br><span class="line">||--items.py</span><br><span class="line">||--piplines.py</span><br><span class="line">||--settings.py</span><br><span class="line">||--spiders/</span><br><span class="line">||--`__init__.py`</span><br></pre></td></tr></table></figure><p>将上一步完成的代码保存为<code>geturls.py</code>放到<code>recruit</code>(与items.py等文件同级)目录下。</p><h4 id="定义items"><a href="#定义items" class="headerlink" title="定义items"></a>定义items</h4><p>先打开网页的职位信息页面看看需要提取哪些信息吧</p><p><img src="http://okzvvfkr0.bkt.clouddn.com/description.jpg"></p><p>除了这些信息外，下面还有职位描述和地址信息等，把这些信息定义为items，即为放置这些信息的容器。<br>打开<code>items.py</code>敲入以下代码</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># -*- coding: utf-8 -*-</span></span><br><span class="line"><span class="keyword">import</span> scrapy</span><br><span class="line"></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">RecruitItem</span>(scrapy.Item):</span><br><span class="line"> position = scrapy.Field()</span><br><span class="line"> company = scrapy.Field()</span><br><span class="line"> salary = scrapy.Field()</span><br><span class="line"> location = scrapy.Field()</span><br><span class="line"> date = scrapy.Field()</span><br><span class="line"> nature = scrapy.Field()</span><br><span class="line"> experience = scrapy.Field()</span><br><span class="line"> education = scrapy.Field()</span><br><span class="line"> number = scrapy.Field()</span><br><span class="line"> category = scrapy.Field()</span><br><span class="line"> description = scrapy.Field()</span><br><span class="line"> address = scrapy.Field()</span><br></pre></td></tr></table></figure><p>保存后退出。</p><h4 id="定义spider"><a href="#定义spider" class="headerlink" title="定义spider"></a>定义spider</h4><p>在<code>spiders</code>文件夹内新建文件<code>recruit_spider.py</code>,这是爬虫的核心文件。<br>首先引入需要的模块</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#coding:utf-8</span></span><br><span class="line"><span class="keyword">import</span> scrapy</span><br><span class="line"><span class="keyword">from</span> recruit.items <span class="keyword">import</span> RecruitItem</span><br><span class="line"><span class="keyword">from</span> recruit.geturls <span class="keyword">import</span> Geturls</span><br><span class="line"><span class="keyword">from</span> scrapy.http <span class="keyword">import</span> Request</span><br></pre></td></tr></table></figure><p>这里要引入前面编写的<code>Geturls类</code>和定义的<code>items</code>.<br>新建类，继承框架的爬虫基类，并注明唯一的爬虫名称</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">RecruitSpider</span>(scrapy.Spider):</span><br><span class="line">name = <span class="string">'recruit'</span></span><br></pre></td></tr></table></figure><p>其余的方法在这个类中编写。第一个方法是</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">start_requests</span>(<span class="params">self</span>):</span><br><span class="line">base_url = <span class="string">"http://sou.zhaopin.com/jobs/searchresult.ashx?jl=%E5%A4%A7%E8%BF%9E&kw=php&sm=0&p=1"</span></span><br><span class="line">geturls = Geturls()</span><br><span class="line">content = geturls.<span class="built_in">open</span>(base_url)</span><br><span class="line">suburls = geturls.distill(content)</span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> url <span class="keyword">in</span> suburls:</span><br><span class="line">url_str = <span class="built_in">str</span>(url)</span><br><span class="line">url_temp = url_str.strip(<span class="string">"'[]'"</span>)</span><br><span class="line"><span class="keyword">yield</span> Request(url_temp, self.parse)</span><br></pre></td></tr></table></figure><p>这是<code>scrapy</code>内置的方法名称，依据名称可知是开启<code>requests</code>执行的函数，这里用到了第一步编写的提取url类，直接使用。需要注意的是传过来的url可能包含多余字符，使用<code>strip()</code>去除。<code>base_url</code>是在智联主页搜索后跳转的链接，可以发现这个链接也是有规律可循的，规律是这样的</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">jl=地点，kw=职位名称，sm=页面状态码(默认=<span class="number">0</span>，可以无视)，p=页码;</span><br></pre></td></tr></table></figure><p>可以根据这个规律，自行创建。地点职位页码，根据自身情况修改即可，或者将页码代入循环自动生成一系列地址逐个爬取，这里就不再拓展了……<br>然后将<code>url</code>传给scrapy内置方法<code>Request()</code>，并且使用<code>parse</code>方法作为回调函数进行筛选。</p><h4 id="爬取信息"><a href="#爬取信息" class="headerlink" title="爬取信息"></a>爬取信息</h4><p>爬取信息更加简单了，根据<code>xpath</code>获取信息位置取得文本，再剔除多余的空格和字符再保存到<code>items</code>内即可。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">parse</span>(<span class="params">self, response</span>):</span><br><span class="line">items = RecruitItem()</span><br><span class="line"></span><br><span class="line">items[<span class="string">'position'</span>] = <span class="string">""</span>.join(response.xpath(<span class="string">'/html/body/div[5]/div[1]/div[1]/h1/text()'</span>).extract()).strip()</span><br><span class="line">items[<span class="string">'company'</span>] = <span class="string">""</span>.join(response.xpath(<span class="string">'/html/body/div[5]/div[1]/div[1]/h2/a/text()'</span>).extract()).strip()</span><br><span class="line">items[<span class="string">'salary'</span>] = <span class="string">""</span>.join(response.xpath(<span class="string">'/html/body/div[6]/div[1]/ul/li[1]/strong/text()'</span>).extract()).strip()</span><br><span class="line">items[<span class="string">'location'</span>] = <span class="string">""</span>.join(response.xpath(<span class="string">'/html/body/div[6]/div[1]/ul/li[2]/strong/a/text()'</span>).extract()).strip()</span><br><span class="line">items[<span class="string">'date'</span>] = <span class="string">""</span>.join(response.xpath(<span class="string">'//*[@id="span4freshdate"]/text()'</span>).extract()).strip()</span><br><span class="line">items[<span class="string">'nature'</span>] = <span class="string">""</span>.join(response.xpath(<span class="string">'/html/body/div[6]/div[1]/ul/li[4]/strong/text()'</span>).extract()).strip()</span><br><span class="line">items[<span class="string">'experience'</span>] = <span class="string">""</span>.join(response.xpath(<span class="string">'/html/body/div[6]/div[1]/ul/li[5]/strong/text()'</span>).extract()).strip()</span><br><span class="line">items[<span class="string">'education'</span>] = <span class="string">""</span>.join(response.xpath(<span class="string">'/html/body/div[6]/div[1]/ul/li[6]/strong/text()'</span>).extract()).strip()</span><br><span class="line">items[<span class="string">'number'</span>] = <span class="string">""</span>.join(response.xpath(<span class="string">'/html/body/div[6]/div[1]/ul/li[7]/strong/text()'</span>).extract()).strip()</span><br><span class="line">items[<span class="string">'category'</span>] = <span class="string">""</span>.join(response.xpath(<span class="string">'/html/body/div[6]/div[1]/ul/li[8]/strong/a/text()'</span>).extract()).strip()</span><br><span class="line">items[<span class="string">'description'</span>] = <span class="string">""</span>.join(response.xpath(<span class="string">'/html/body/div[6]/div[1]/div[1]/div/div[1]/p[1]/text()'</span>).extract()).strip()</span><br><span class="line">items[<span class="string">'address'</span>] = <span class="string">""</span>.join(response.xpath(<span class="string">'/html/body/div[6]/div[1]/div[1]/div/div[1]/h2/text()'</span>).extract()).strip()</span><br><span class="line"></span><br><span class="line"><span class="keyword">return</span> items</span><br><span class="line"></span><br></pre></td></tr></table></figure><h4 id="定义pipelines"><a href="#定义pipelines" class="headerlink" title="定义pipelines"></a>定义pipelines</h4><p><code>pipelines</code>模块是将<code>items</code>内的信息做过滤，持久化或其他工作的。<br>这里的代码跟之前<code>SCRAPY简单入门</code>中几乎一样，就不再赘述了。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># -*- coding: utf-8 -*-</span></span><br><span class="line"><span class="keyword">import</span> json </span><br><span class="line"><span class="keyword">import</span> codecs</span><br><span class="line"><span class="keyword">from</span> collections <span class="keyword">import</span> OrderedDict</span><br><span class="line"></span><br><span class="line"><span class="keyword">class</span> <span class="title class_">RecruitPipeline</span>(<span class="title class_ inherited__">object</span>):</span><br><span class="line"><span class="keyword">def</span> <span class="title function_">__init__</span>(<span class="params">self</span>):</span><br><span class="line">self.file = codecs.<span class="built_in">open</span>(<span class="string">'recruit.json'</span>,<span class="string">'wb'</span>,encoding=<span class="string">'utf-8'</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">process_item</span>(<span class="params">self, item, spider</span>):</span><br><span class="line">line = json.dumps(OrderedDict(item))</span><br><span class="line">self.file.write(line.decode(<span class="string">"unicode_escape"</span>))</span><br><span class="line">self.file.write(<span class="string">"\n"</span>)</span><br><span class="line"><span class="keyword">return</span> item</span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">close_spider</span>(<span class="params">self, spider</span>):</span><br><span class="line">self.file.close()</span><br></pre></td></tr></table></figure><p>将<code>items</code>内的信息写入json文件<br>不要忘记在<code>setting.py</code>文件中，取消以下几行的注释,并修改成自己项目的名字。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">ITEM_PIPELINES = {</span><br><span class="line"> <span class="string">'recruit.pipelines.RecruitPipeline'</span>: <span class="number">300</span>,</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>OK，所有准备工作完成！打开命令行键入</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">scrapy crawl recruit</span><br></pre></td></tr></table></figure><p>见证奇迹的发生吧～</p><h3 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h3><p>这个小项目整体来说并不难，应该说麻烦多一些。其实数据导出之后，才做好了一个开头工作，本打算将这些数据做成一个可视化的图表，这样才能更直观的感受到信息的价值。数据处理稍微查了一下也是个大坑，更何况这种中文信息处理，想想就有点头疼。话不多说，继续学习，待将数据处理完成后写一篇新的文章。</p>]]></content>
<summary type="html">新的一年准备换一个新工作，如何快速获取大量的职位信息呢？干脆编写一个爬虫好了，顺带巩固一下知识。</summary>
<category term="Python" scheme="https://catinsides.github.io/tags/Python/"/>
</entry>
<entry>
<title>如何编写自己的MVC框架</title>
<link href="https://catinsides.github.io/2017/01/26/%E5%A6%82%E4%BD%95%E7%BC%96%E5%86%99%E8%87%AA%E5%B7%B1%E7%9A%84MVC%E6%A1%86%E6%9E%B6/"/>
<id>https://catinsides.github.io/2017/01/26/%E5%A6%82%E4%BD%95%E7%BC%96%E5%86%99%E8%87%AA%E5%B7%B1%E7%9A%84MVC%E6%A1%86%E6%9E%B6/</id>
<published>2017-01-26T06:22:56.000Z</published>
<updated>2024-08-14T08:42:02.743Z</updated>
<content type="html"><![CDATA[<p>MVC是一种软件设计模式。顾名思义，这些字母分别代表着模型（Model）、视图（View）和控制器（Controller）。PHP中的MVC旨在分离业务与逻辑层，使得开发过程中的结构清晰，大大提高了效率。当下有很多流行的PHP框架均使用了MVC模式。如果是新手入门，查看这些框架的源代码肯定会一头雾水，毫无思路。上手新框架等于重新开始学习，这必然会消耗不必要的精力和时间。而学习MVC框架最好的方式就是自己动手写一个框架，既学习了设计模式，又巩固了编程能力，何乐而不为。更重要的是，在开发过程中能够将自己的想法融入到框架中去，从而实现各种功能。</p><h3 id="准备工作"><a href="#准备工作" class="headerlink" title="准备工作"></a>准备工作</h3><p>这篇文章是我<a href="https://www.google.com.hk/#newwindow=1&safe=active&q=code+your+own+php+mvc+framework">Google</a>学习很多文章后的总结。将各路大神的代码汇总，然后择取出MVC必要的元素，根据个人需要编写出的小型框架，而非使用Composer组装出的框架。这个框架包含以下特点：</p><ul><li>MVC结构</li><li>单一入口</li><li>自动加载类</li><li>数据库封装</li></ul><p>根据这个思路，开始coding.</p><h3 id="MVC流程"><a href="#MVC流程" class="headerlink" title="MVC流程"></a>MVC流程</h3><p>日常上网过程中可以简化理解为人与服务器中数据库交互的过程。<br>每一次网页上的请求发送到服务器，服务端将请求通过控制器处理后，再传送到模型。<br>模型内包含业务逻辑，负责处理如何与数据库交换数据，数据处理完成后，发送到视图。<br>视图将传来的数据渲染后，最终返还用户。这样便完成了一次请求。<br>熟悉MVC流程后，就有了框架的宏观印象，方便理清结构。</p><h3 id="编写框架"><a href="#编写框架" class="headerlink" title="编写框架"></a>编写框架</h3><h4 id="建立目录"><a href="#建立目录" class="headerlink" title="建立目录"></a>建立目录</h4><p>首先，在项目目录下建立如下文件和文件夹:</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">|--application</span><br><span class="line">| |--controllers</span><br><span class="line">| |--models</span><br><span class="line">| |--views</span><br><span class="line">|--config</span><br><span class="line">|--framework</span><br><span class="line">|--.htaccess</span><br><span class="line">|--index.php</span><br></pre></td></tr></table></figure><p><code>application</code> 内放置用户编写的MVC文件；<br><code>config</code> 内放置数据库设置文件；<br><code>framework</code> 内放置框架的核心文件；<br><code>.htaccess</code> 为重定向文件；<br><code>index.php</code> 为入口文件。</p><h4 id="重定向"><a href="#重定向" class="headerlink" title="重定向"></a>重定向</h4><p><code>.htaccess</code>是服务器的配置文件，它用来将所有的请求转到入口文件处理。这样一来，网页程序便有了单一入口，掌控用户请求，避免熊孩子的无理取闹（笑。还可以生成对搜索引擎友好的URL。配置内容如下：<br>Apache服务器：</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">RewriteEngine on</span><br><span class="line"> RewriteCond %{REQUEST_FILENAME} !-f</span><br><span class="line"> RewriteCond %{REQUEST_FILENAME} !-d</span><br><span class="line"> RewriteRule ^(.+)$ index.php/$1 [L]</span><br></pre></td></tr></table></figure><p>nginx服务器：</p><figure class="highlight nginx"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="section">location</span> / {</span><br><span class="line"> if(!-<span class="attribute">e</span> <span class="variable">$request_filename</span>){</span><br><span class="line"> <span class="attribute">rewrite</span><span class="regexp"> ^(.+)$</span> /index.php/<span class="variable">$1</span> <span class="literal">break</span>;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h4 id="入口文件"><a href="#入口文件" class="headerlink" title="入口文件"></a>入口文件</h4><p>打开同目录下的index.php文件，输入以下代码：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"><?php</span></span><br><span class="line"><span class="keyword">require</span> <span class="string">'./framework/framework.php'</span>;</span><br></pre></td></tr></table></figure><p>内容很简单，就是包含核心文件夹内的入口文件。其实在这里可以定义一些预定义常量方便调用，例如：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="title function_ invoke__">define</span>(<span class="string">'APP_PATH'</span>, <span class="keyword">__DIR__</span>.<span class="string">'/'</span>);</span><br><span class="line"><span class="title function_ invoke__">define</span>(<span class="string">'APP_DEBUG'</span>, <span class="literal">true</span>);</span><br><span class="line"><span class="title function_ invoke__">define</span>(<span class="string">'APP_URL'</span>, <span class="string">'http://localhost'</span>);</span><br></pre></td></tr></table></figure><h4 id="核心文件"><a href="#核心文件" class="headerlink" title="核心文件"></a>核心文件</h4><p>外入口文件(index.php)将请求传递给核心文件的内入口文件(framework.php)后，接下来的工作由核心文件完成。framework.php的内容如下：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"><?php</span></span><br><span class="line"><span class="title function_ invoke__">defined</span>(<span class="string">'FRAME_PATH'</span>) <span class="keyword">or</span> <span class="title function_ invoke__">define</span>(<span class="string">'FRAME_PATH'</span>, <span class="keyword">__DIR__</span>.<span class="string">'/'</span>);</span><br><span class="line"><span class="title function_ invoke__">defined</span>(<span class="string">'APP_PATH'</span>) <span class="keyword">or</span> <span class="title function_ invoke__">define</span>(<span class="string">'APP_PATH'</span>, <span class="title function_ invoke__">dirname</span>(<span class="variable">$_SERVER</span>[<span class="string">'SCRIPT_FILENAME'</span>]).<span class="string">'/'</span>);</span><br><span class="line"><span class="title function_ invoke__">defined</span>(<span class="string">'APP_DEBUG'</span>) <span class="keyword">or</span> <span class="title function_ invoke__">define</span>(<span class="string">'APP_DEBUG'</span>, <span class="literal">false</span>);</span><br><span class="line"><span class="title function_ invoke__">defined</span>(<span class="string">'CONFIG_PATH'</span>) <span class="keyword">or</span> <span class="title function_ invoke__">define</span>(<span class="string">'CONFIG_PATH'</span>, APP_PATH.<span class="string">'config/'</span>);</span><br><span class="line"><span class="title function_ invoke__">defined</span>(<span class="string">'RUNTIME_PATH'</span>) <span class="keyword">or</span> <span class="title function_ invoke__">define</span>(<span class="string">'RUNTIME_PATH'</span>, APP_PATH.<span class="string">'runtime/'</span>);</span><br><span class="line"></span><br><span class="line"><span class="keyword">require</span> APP_PATH . <span class="string">'config/config.php'</span>;</span><br><span class="line"></span><br><span class="line"><span class="keyword">require</span> FRAME_PATH . <span class="string">'Core.php'</span>;</span><br><span class="line"></span><br><span class="line"><span class="variable">$framework</span> = <span class="keyword">new</span> <span class="title class_">Core</span>;</span><br><span class="line"><span class="variable">$framework</span>-><span class="title function_ invoke__">run</span>();</span><br></pre></td></tr></table></figure><p>同样，在这里进行预定义常量和包含文件的工作，并实例化核心文件，让网页程序得以运行。题外话：看了一些外国人写的教程，大部分都喜欢用<code>$framework->bootstrap()</code>, 而这个<code>bootstrap()</code>并非代表着前端框架Bootstrap（原谅我脑洞大，第一个想到的就是这个），而是一种命名习惯吧。<br>从上述代码可以看出，包含了两个文件，其中<code>config.php</code>是数据库配置文件，要把它存放在<code>config</code>文件夹内，其内容如下：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"><?php</span></span><br><span class="line"><span class="title function_ invoke__">define</span>(<span class="string">'DB_NAME'</span>, <span class="string">'test_db'</span>);</span><br><span class="line"><span class="title function_ invoke__">define</span>(<span class="string">'DB_USER'</span>, <span class="string">'root'</span>);</span><br><span class="line"><span class="title function_ invoke__">define</span>(<span class="string">'DB_PASSWORD'</span>, <span class="string">'1234'</span>);</span><br><span class="line"><span class="title function_ invoke__">define</span>(<span class="string">'DB_HOST'</span>, <span class="string">'localhost'</span>);</span><br></pre></td></tr></table></figure><p>根据个人情况修改即可。<br>回到<code>framework</code>文件夹，建立<code>Core.php</code>，这是框架核心文件，内容如下：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"><?php</span></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Core</span></span></span><br><span class="line"><span class="class"></span>{</span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">run</span>(<span class="params"></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="title function_ invoke__">spl_autoload_register</span>(<span class="keyword">array</span>(<span class="variable">$this</span>, <span class="string">'loadClass'</span>));</span><br><span class="line"> <span class="variable language_">$this</span>-><span class="title function_ invoke__">setReporting</span>();</span><br><span class="line"> <span class="variable language_">$this</span>-><span class="title function_ invoke__">removeMagicQuotes</span>();</span><br><span class="line"> <span class="variable language_">$this</span>-><span class="title function_ invoke__">unregisterGlobals</span>();</span><br><span class="line"> <span class="variable language_">$this</span>-><span class="title function_ invoke__">route</span>();</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">route</span>(<span class="params"></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="variable">$controllerName</span> = <span class="string">'Index'</span>;</span><br><span class="line"> <span class="variable">$action</span> = <span class="string">'index'</span>;</span><br><span class="line"> <span class="variable">$param</span> = <span class="keyword">array</span>();</span><br><span class="line"> <span class="variable">$url</span> = <span class="keyword">isset</span>(<span class="variable">$_GET</span>[<span class="string">'url'</span>]) ? <span class="variable">$_GET</span>[<span class="string">'url'</span>] : <span class="literal">false</span>;</span><br><span class="line"> <span class="keyword">if</span> (<span class="variable">$url</span>) { </span><br><span class="line"> <span class="variable">$urlArray</span> = <span class="title function_ invoke__">explode</span>(<span class="string">'/'</span>, <span class="variable">$url</span>);</span><br><span class="line"> <span class="variable">$urlArray</span> = <span class="title function_ invoke__">array_filter</span>(<span class="variable">$urlArray</span>);</span><br><span class="line"> <span class="variable">$controllerName</span> = <span class="title function_ invoke__">ucfirst</span>(<span class="variable">$urlArray</span>[<span class="number">0</span>]);</span><br><span class="line"> <span class="title function_ invoke__">array_shift</span>(<span class="variable">$urlArray</span>);</span><br><span class="line"> <span class="variable">$action</span> = <span class="variable">$urlArray</span> ? <span class="variable">$urlArray</span>[<span class="number">0</span>] : <span class="string">'index'</span>;</span><br><span class="line"> <span class="title function_ invoke__">array_shift</span>(<span class="variable">$urlArray</span>);</span><br><span class="line"> <span class="variable">$param</span> = <span class="variable">$urlArray</span> ? <span class="variable">$urlArray</span> : <span class="keyword">array</span>();</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="variable">$controller</span> = <span class="variable">$controllerName</span> . <span class="string">'Controller'</span>;</span><br><span class="line"> <span class="variable">$dispatch</span> = <span class="keyword">new</span> <span class="variable">$controller</span>(<span class="variable">$controllerName</span>, <span class="variable">$action</span>);</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">if</span> ((<span class="keyword">int</span>)<span class="title function_ invoke__">method_exists</span>(<span class="variable">$controller</span>, <span class="variable">$action</span>)) {</span><br><span class="line"> <span class="title function_ invoke__">call_user_func_array</span>(<span class="keyword">array</span>(<span class="variable">$dispatch</span>, <span class="variable">$action</span>), <span class="variable">$param</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">exit</span>(<span class="variable">$controller</span> . <span class="string">"控制器不存在"</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">setReporting</span>(<span class="params"></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="keyword">if</span> (APP_DEBUG === <span class="literal">true</span>) {</span><br><span class="line"> <span class="title function_ invoke__">error_reporting</span>(E_ALL);</span><br><span class="line"> <span class="title function_ invoke__">ini_set</span>(<span class="string">'display_errors'</span>,<span class="string">'On'</span>);</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="title function_ invoke__">error_reporting</span>(E_ALL);</span><br><span class="line"> <span class="title function_ invoke__">ini_set</span>(<span class="string">'display_errors'</span>,<span class="string">'Off'</span>);</span><br><span class="line"> <span class="title function_ invoke__">ini_set</span>(<span class="string">'log_errors'</span>, <span class="string">'On'</span>);</span><br><span class="line"> <span class="title function_ invoke__">ini_set</span>(<span class="string">'error_log'</span>, RUNTIME_PATH. <span class="string">'logs/error.log'</span>);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">stripSlashesDeep</span>(<span class="params"><span class="variable">$value</span></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="variable">$value</span> = <span class="title function_ invoke__">is_array</span>(<span class="variable">$value</span>) ? <span class="title function_ invoke__">array_map</span>(<span class="keyword">array</span>(<span class="variable">$this</span>, <span class="string">'stripSlashesDeep'</span>), <span class="variable">$value</span>) : <span class="title function_ invoke__">stripslashes</span>(<span class="variable">$value</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="variable">$value</span>;</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">removeMagicQuotes</span>(<span class="params"></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="keyword">if</span> (<span class="title function_ invoke__">get_magic_quotes_gpc</span>()) {</span><br><span class="line"> <span class="variable">$_GET</span> = <span class="keyword">isset</span>(<span class="variable">$_GET</span>) ? <span class="variable language_">$this</span>-><span class="title function_ invoke__">stripSlashesDeep</span>(<span class="variable">$_GET</span> ) : <span class="string">''</span>;</span><br><span class="line"> <span class="variable">$_POST</span> = <span class="keyword">isset</span>(<span class="variable">$_POST</span>) ? <span class="variable language_">$this</span>-><span class="title function_ invoke__">stripSlashesDeep</span>(<span class="variable">$_POST</span> ) : <span class="string">''</span>;</span><br><span class="line"> <span class="variable">$_COOKIE</span> = <span class="keyword">isset</span>(<span class="variable">$_COOKIE</span>) ? <span class="variable language_">$this</span>-><span class="title function_ invoke__">stripSlashesDeep</span>(<span class="variable">$_COOKIE</span>) : <span class="string">''</span>;</span><br><span class="line"> <span class="variable">$_SESSION</span> = <span class="keyword">isset</span>(<span class="variable">$_SESSION</span>) ? <span class="variable language_">$this</span>-><span class="title function_ invoke__">stripSlashesDeep</span>(<span class="variable">$_SESSION</span>) : <span class="string">''</span>;</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">unregisterGlobals</span>(<span class="params"></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="keyword">if</span> (<span class="title function_ invoke__">ini_get</span>(<span class="string">'register_globals'</span>)) {</span><br><span class="line"> <span class="variable">$array</span> = <span class="keyword">array</span>(<span class="string">'_SESSION'</span>, <span class="string">'_POST'</span>, <span class="string">'_GET'</span>, <span class="string">'_COOKIE'</span>, <span class="string">'_REQUEST'</span>, <span class="string">'_SERVER'</span>, <span class="string">'_ENV'</span>, <span class="string">'_FILES'</span>);</span><br><span class="line"> <span class="keyword">foreach</span> (<span class="variable">$array</span> <span class="keyword">as</span> <span class="variable">$value</span>) {</span><br><span class="line"> <span class="keyword">foreach</span> (<span class="variable">$GLOBALS</span>[<span class="variable">$value</span>] <span class="keyword">as</span> <span class="variable">$key</span> => <span class="variable">$var</span>) {</span><br><span class="line"> <span class="keyword">if</span> (<span class="variable">$var</span> === <span class="variable">$GLOBALS</span>[<span class="variable">$key</span>]) {</span><br><span class="line"> <span class="keyword">unset</span>(<span class="variable">$GLOBALS</span>[<span class="variable">$key</span>]);</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">public</span> <span class="built_in">static</span> <span class="function"><span class="keyword">function</span> <span class="title">loadClass</span>(<span class="params"><span class="variable">$class</span></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="variable">$frameworks</span> = FRAME_PATH . <span class="variable">$class</span> . <span class="string">'.class.php'</span>;</span><br><span class="line"> <span class="variable">$controllers</span> = APP_PATH . <span class="string">'application/controllers/'</span> . <span class="variable">$class</span> . <span class="string">'.class.php'</span>;</span><br><span class="line"> <span class="variable">$models</span> = APP_PATH . <span class="string">'application/models/'</span> . <span class="variable">$class</span> . <span class="string">'.class.php'</span>;</span><br><span class="line"> <span class="keyword">if</span> (<span class="title function_ invoke__">file_exists</span>(<span class="variable">$frameworks</span>)) {</span><br><span class="line"> <span class="keyword">include</span> <span class="variable">$frameworks</span>;</span><br><span class="line"> } <span class="keyword">elseif</span> (<span class="title function_ invoke__">file_exists</span>(<span class="variable">$controllers</span>)) {</span><br><span class="line"> <span class="keyword">include</span> <span class="variable">$controllers</span>;</span><br><span class="line"> } <span class="keyword">elseif</span> (<span class="title function_ invoke__">file_exists</span>(<span class="variable">$models</span>)) {</span><br><span class="line"> <span class="keyword">include</span> <span class="variable">$models</span>;</span><br><span class="line"> } <span class="keyword">else</span> {</span><br><span class="line"> <span class="keyword">echo</span> <span class="string">"控制器类或模型类不存在！"</span></span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>从<code>run()</code>中可以理解这一核心文件的运行机制，首先<code>spl_autoload_register()</code>用于自动加载类文件，在以后的类实例化中，可以方便的直接调用，而不是一个一个的require;接下来<code>setReporting()</code>用来开启调试模式，并记录到日志中;<code>removeMagicQuotes()</code>用于移除敏感字符;<code>unregisterGlobals()</code>用于检测并移除系统全局变量，避免如<code>_GET</code>不到变量的问题。准备工作完成后，<code>route()</code>方法截取URL，将形如：<br><code>localhost/?url=controller/action/parameters</code><br>的URL分离为<code>controller</code>,<code>action</code>和<code>parameters</code>，其中<code>controller</code>即<code>application</code>文件夹下用户自定义的控制器类文件，<code>action</code>为相应类文件内的方法，<code>parameters</code>为传入方法内的变量。实例化控制器后，并调用view方法。<br>这里的URL有点古怪，因为使用是<code>GET</code>方法获取的。如果想去掉<code>?url</code>也未尝不可，使用<code>$_SERVER['PATH_INFO']</code>可获取<code>index.php</code>后的内容，以<code>/</code>分离可获取控制器和方法；而<code>？</code>后的内容可以使用<code>$_SERVER['QUERY_STRING']</code>获取。<code>PATHINFO</code>的URL模式对搜索引擎更加友好。</p><h4 id="MVC基类"><a href="#MVC基类" class="headerlink" title="MVC基类"></a>MVC基类</h4><p>接下来，在核心文件夹内创建<code>Controller.class.php</code>,<code>Model.class.php</code>和<code>View.class.php</code>三个基类文件，用户创建的子类文件需要继承基类文件，所以可以把一些常用的方法写入到这个文件内，日后使用时直接调用即可。同时，因为这些基类文件负责着整体的MVC流程，所以其中要包含一些总体调度方法。<br><code>Controller.class.php</code>内容如下：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"><?php</span> </span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Controller</span></span></span><br><span class="line"><span class="class"></span>{</span><br><span class="line"> <span class="keyword">protected</span> <span class="variable">$_controller</span>;</span><br><span class="line"> <span class="keyword">protected</span> <span class="variable">$_action</span>;</span><br><span class="line"> <span class="keyword">protected</span> <span class="variable">$_view</span>;</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">__construct</span>(<span class="params"><span class="variable">$controller</span>, <span class="variable">$action</span></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="variable language_">$this</span>->_controller = <span class="variable">$controller</span>;</span><br><span class="line"> <span class="variable language_">$this</span>->_action = <span class="variable">$action</span>;</span><br><span class="line"> <span class="variable language_">$this</span>->_view = <span class="keyword">new</span> <span class="title class_">View</span>(<span class="variable">$controller</span>, <span class="variable">$action</span>);</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">assign</span>(<span class="params"><span class="variable">$name</span>, <span class="variable">$value</span></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="variable language_">$this</span>->_view-><span class="title function_ invoke__">assign</span>(<span class="variable">$name</span>, <span class="variable">$value</span>);</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">render</span>(<span class="params"></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="variable language_">$this</span>->_view-><span class="title function_ invoke__">render</span>();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>从代码内容可知，控制器基类用于分配变量和传递变量给视图，所以在控制器子类中，可以直接使用<code>$this->render()</code> 进行渲染。<br><code>Model.class.php</code>内容如下：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"><?php</span></span><br><span class="line"> <span class="class"><span class="keyword">class</span> <span class="title">Model</span> <span class="keyword">extends</span> <span class="title">Sql</span></span></span><br><span class="line"><span class="class"> </span>{</span><br><span class="line"> <span class="keyword">protected</span> <span class="variable">$_model</span>;</span><br><span class="line"> <span class="keyword">protected</span> <span class="variable">$_table</span>;</span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">__construct</span>(<span class="params"></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="variable language_">$this</span>-><span class="title function_ invoke__">connect</span>(DB_HOST, DB_USER, DB_PASSWORD, DB_NAME);</span><br><span class="line"> <span class="variable language_">$this</span>->_model = <span class="title function_ invoke__">get_class</span>(<span class="variable">$this</span>);</span><br><span class="line"> <span class="variable language_">$this</span>->_model = <span class="title function_ invoke__">substr</span>(<span class="variable">$this</span>->_model, <span class="number">0</span>, -<span class="number">5</span>);</span><br><span class="line"> <span class="variable language_">$this</span>->_table = <span class="title function_ invoke__">strtolower</span>(<span class="variable">$this</span>->_model);</span><br><span class="line"> } </span><br><span class="line"> }</span><br></pre></td></tr></table></figure><p>此类继承自数据库基类<code>Sql.class.php</code>,其内容如下：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"><?php</span> </span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Sql</span></span></span><br><span class="line"><span class="class"></span>{</span><br><span class="line"> <span class="keyword">protected</span> <span class="variable">$_dbHandle</span>;</span><br><span class="line"> <span class="keyword">protected</span> <span class="variable">$_result</span>;</span><br><span class="line"> <span class="keyword">private</span> <span class="variable">$filter</span> = <span class="string">''</span>;</span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">connect</span>(<span class="params"><span class="variable">$host</span>, <span class="variable">$user</span>, <span class="variable">$pass</span>, <span class="variable">$dbname</span></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="variable">$dsn</span> = <span class="title function_ invoke__">sprintf</span>(<span class="string">"mysql:host=%s;dbname=%s;charset=utf8"</span>, <span class="variable">$host</span>, <span class="variable">$dbname</span>);</span><br><span class="line"> <span class="variable">$option</span> = <span class="keyword">array</span>(PDO::<span class="variable constant_">ATTR_DEFAULT_FETCH_MODE</span> => PDO::<span class="variable constant_">FETCH_ASSOC</span>);</span><br><span class="line"> <span class="variable language_">$this</span>->_dbHandle = <span class="keyword">new</span> <span class="title function_ invoke__">PDO</span>(<span class="variable">$dsn</span>, <span class="variable">$user</span>, <span class="variable">$pass</span>, <span class="variable">$option</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (PDOException <span class="variable">$e</span>) {</span><br><span class="line"> <span class="keyword">exit</span>(<span class="string">'错误: '</span> . <span class="variable">$e</span>-><span class="title function_ invoke__">getMessage</span>());</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">select</span>(<span class="params"><span class="variable">$id</span></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="variable">$sql</span> = <span class="title function_ invoke__">sprintf</span>(<span class="string">"select * from `%s` where `id` = '%s'"</span>, <span class="variable">$this</span>->_table, <span class="variable">$id</span>);</span><br><span class="line"> <span class="variable">$sth</span> = <span class="variable language_">$this</span>->_dbHandle-><span class="title function_ invoke__">prepare</span>(<span class="variable">$sql</span>);</span><br><span class="line"> <span class="variable">$sth</span>-><span class="title function_ invoke__">execute</span>();</span><br><span class="line"> <span class="keyword">return</span> <span class="variable">$sth</span>-><span class="title function_ invoke__">fetch</span>();</span><br><span class="line"> } </span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>这里只封装了普通查询的方法，可以以这个方法为模板编写其他的CURD方法。再以后的使用过程中，就可以直接使用封装后的方法操作数据库<br><code>View.class.php</code>内容如下：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">View</span></span></span><br><span class="line"><span class="class"></span>{</span><br><span class="line"> <span class="keyword">protected</span> <span class="variable">$variables</span> = <span class="keyword">array</span>();</span><br><span class="line"> <span class="keyword">protected</span> <span class="variable">$_controller</span>;</span><br><span class="line"> <span class="keyword">protected</span> <span class="variable">$_action</span>;</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">function</span> <span class="title">__construct</span>(<span class="params"><span class="variable">$controller</span>, <span class="variable">$action</span></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="variable language_">$this</span>->_controller = <span class="variable">$controller</span>;</span><br><span class="line"> <span class="variable language_">$this</span>->_action = <span class="variable">$action</span>;</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">assign</span>(<span class="params"><span class="variable">$name</span>, <span class="variable">$value</span></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="variable language_">$this</span>->variables[<span class="variable">$name</span>] = <span class="variable">$value</span>;</span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">render</span>(<span class="params"></span>)</span></span><br><span class="line"><span class="function"> </span>{</span><br><span class="line"> <span class="title function_ invoke__">extract</span>(<span class="variable">$this</span>->variables);</span><br><span class="line"> <span class="comment">//加载自定义模板</span></span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>这样，框架的核心部分就完成了。测试框架时，像使用其他框架一样，在<code>application</code>文件夹内的子文件夹分别建立用户类文件，并继承基类文件，写入测试方法和语句即可。</p><h3 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h3><p>当我翻阅了很多编写入门级MVC框架教程之后，发现很多代码内容都是一样的。比如截取URL和自动加载类的方法。查看这些源码很容易理解其原理和过程。由于目前能力有限，我还不能倒背如流般的coding出这些代码，只得借鉴大神的代码，让自己学习和巩固。在开发过程中，也可以使用其他人造好的轮子，利用<code>Composer</code>下载到项目文件夹内，然后根据命名空间规则引用，就可以使用了。这样便极大的加快了项目的开发速度。但是我不喜欢黑箱操作，单纯的调用API反而更容易让人摸不着北。创造，才是一件令人开心的事。</p>]]></content>
<summary type="html">MVC是当下Web开发中流行的设计模式。从自己动手编写框架开始学习，能够有效的学习MVC结构和提高自身的面向对象编程能力。</summary>
<category term="PHP" scheme="https://catinsides.github.io/tags/PHP/"/>
</entry>
<entry>
<title>Leetcode.Database 题解笔记（下）</title>
<link href="https://catinsides.github.io/2016/12/26/Leetcode.Database-%E9%A2%98%E8%A7%A3%E7%AC%94%E8%AE%B0%EF%BC%88%E4%B8%8B%EF%BC%89/"/>
<id>https://catinsides.github.io/2016/12/26/Leetcode.Database-%E9%A2%98%E8%A7%A3%E7%AC%94%E8%AE%B0%EF%BC%88%E4%B8%8B%EF%BC%89/</id>
<published>2016-12-26T07:55:44.000Z</published>
<updated>2024-08-14T08:42:02.740Z</updated>
<content type="html"><![CDATA[<blockquote><p>Leetcode是一个在线编程网站，收录了算法，数据库等各种题目。同样有很多优秀的算法解法，能够有效的提高姿势水平。</p></blockquote><p>这篇文章承接上一篇，将Leetcode.Database中的后6道题的解法及笔记记录下来。</p><h3 id="183-Customers-Who-Never-Order"><a href="#183-Customers-Who-Never-Order" class="headerlink" title="183.Customers Who Never Order"></a>183.<a href="https://leetcode.com/problems/customers-who-never-order/">Customers Who Never Order</a></h3><p>题目要求为查找出没有订单的客户。<br>解题思路为使用子查询和NOT IN.</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> Name <span class="keyword">AS</span> Customers </span><br><span class="line"><span class="keyword">FROM</span> Customers c </span><br><span class="line"><span class="keyword">WHERE</span> c.id </span><br><span class="line"><span class="keyword">NOT</span> <span class="keyword">IN</span> </span><br><span class="line"> (<span class="keyword">SELECT</span> CustomerId <span class="keyword">FROM</span> Orders);</span><br></pre></td></tr></table></figure><p>这里需要注意的是要在NAME后添加<code>AS Customers</code>.</p><h3 id="184-Department-Highest-Salary"><a href="#184-Department-Highest-Salary" class="headerlink" title="184.Department Highest Salary"></a>184.<a href="https://leetcode.com/problems/department-highest-salary/">Department Highest Salary</a></h3><p>题目要求为联合员工表和部门表并从中查找出收入最高的员工。<br>解题思路为使用正确的子查询和过滤条件，</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> d.Name <span class="keyword">AS</span> Department, e.Name <span class="keyword">AS</span> Employee, t.Salary </span><br><span class="line"><span class="keyword">FROM</span> Employee e </span><br><span class="line"><span class="keyword">INNER</span> <span class="keyword">JOIN</span> </span><br><span class="line"> (<span class="keyword">SELECT</span> DepartmentId, <span class="built_in">MAX</span>(Salary) <span class="keyword">AS</span> Salary </span><br><span class="line"> <span class="keyword">FROM</span> Employee </span><br><span class="line"> <span class="keyword">GROUP</span> <span class="keyword">BY</span> DepartmentId) t</span><br><span class="line"><span class="keyword">USING</span> (DepartmentId, Salary)</span><br><span class="line"><span class="keyword">INNER</span> <span class="keyword">JOIN</span> Department d</span><br><span class="line"><span class="keyword">ON</span> d.id <span class="operator">=</span> t.DepartmentId</span><br></pre></td></tr></table></figure><p>这里最容易混淆的是SQL语句的执行顺序，简单做个笔记：<br>SQL语句执行时，每个步骤都会产生一个虚拟表用于下一步骤的输入，虚拟表对外不可用，只会将最终的查询结果返回。如果查询语句中没有某一个子句，则会跳过。<br>顺序结构如下，小括号内为执行次序：</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">(<span class="number">8</span>) <span class="keyword">SELECT</span> (<span class="number">9</span>)<span class="keyword">DISTINCT</span> (<span class="number">11</span>)<span class="operator"><</span>Top Num<span class="operator">></span> <span class="operator"><</span><span class="keyword">select</span> list<span class="operator">></span></span><br><span class="line">(<span class="number">1</span>) <span class="keyword">FROM</span> [left_table]</span><br><span class="line">(<span class="number">3</span>) <span class="operator"><</span>join_type<span class="operator">></span> <span class="keyword">JOIN</span> <span class="operator"><</span>right_table<span class="operator">></span></span><br><span class="line">(<span class="number">2</span>) <span class="keyword">ON</span> <span class="operator"><</span>join_condition<span class="operator">></span></span><br><span class="line">(<span class="number">4</span>) <span class="keyword">WHERE</span> <span class="operator"><</span>where_condition<span class="operator">></span></span><br><span class="line">(<span class="number">5</span>) <span class="keyword">GROUP</span> <span class="keyword">BY</span> <span class="operator"><</span>group_by_list<span class="operator">></span></span><br><span class="line">(<span class="number">6</span>) <span class="keyword">WITH</span> <span class="operator"><</span><span class="keyword">CUBE</span> <span class="operator">|</span> <span class="keyword">RollUP</span><span class="operator">></span></span><br><span class="line">(<span class="number">7</span>) <span class="keyword">HAVING</span> <span class="operator"><</span>having_condition<span class="operator">></span></span><br><span class="line">(<span class="number">10</span>)<span class="keyword">ORDER</span> <span class="keyword">BY</span> <span class="operator"><</span>order_by_list<span class="operator">></span></span><br></pre></td></tr></table></figure><h3 id="185-Department-Top-Three-Salaries"><a href="#185-Department-Top-Three-Salaries" class="headerlink" title="185.Department Top Three Salaries"></a>185.<a href="https://leetcode.com/problems/department-top-three-salaries/">Department Top Three Salaries</a></h3><p>题目要求为查询每个部门前三名收入最高的员工。<br>解题思路，首先复制Employee表e1，原表为e，在同一部门内选一人与其他员工薪水进行比较，将大于此人的人员数量计数。如果数量为0，则此人为薪水最高者；数量为1，则为第二高者；以此类推……将数量小于3（前三名）的员工，再与部门表做INNER JOIN，最终得出结果。<br>查询语句如下：</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> d.Name <span class="keyword">AS</span> Department, e.Name <span class="keyword">AS</span> Employee, e.Salary <span class="keyword">AS</span> Salary </span><br><span class="line"><span class="keyword">FROM</span> Employee e </span><br><span class="line"><span class="keyword">INNER</span> <span class="keyword">JOIN</span> Department d </span><br><span class="line"><span class="keyword">ON</span> e.DepartmentId <span class="operator">=</span> d.Id </span><br><span class="line"><span class="keyword">WHERE</span> <span class="number">3</span> <span class="operator">></span> </span><br><span class="line"> (</span><br><span class="line"> <span class="keyword">SELECT</span> <span class="built_in">COUNT</span>(<span class="keyword">DISTINCT</span> e1.Salary) </span><br><span class="line"> <span class="keyword">FROM</span> Employee e1 </span><br><span class="line"> <span class="keyword">WHERE</span> e1.Salary <span class="operator">></span> e.Salary </span><br><span class="line"> <span class="keyword">AND</span> e1.DepartmentId <span class="operator">=</span> e.DepartmentId</span><br><span class="line"> )</span><br></pre></td></tr></table></figure><p>这道题查询方式很巧妙，需要注意的是将查询方法实践到语句中，同时注意SQL语句的执行顺序。</p><h3 id="186-Delete-Duplicate-Emails"><a href="#186-Delete-Duplicate-Emails" class="headerlink" title="186.Delete Duplicate Emails"></a>186.<a href="https://leetcode.com/problems/delete-duplicate-emails/">Delete Duplicate Emails</a></h3><p>题目要求为删除重复的Email.<br>解题思路为复制临时表，取两者交集，删除掉Email相同Id不同的行（取大于小于皆可）。<br>查询语句如下：</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">DELETE</span> p1 </span><br><span class="line"><span class="keyword">FROM</span> Person p1 </span><br><span class="line"><span class="keyword">INNER</span> <span class="keyword">JOIN</span> Person p2</span><br><span class="line"><span class="keyword">WHERE</span> p1.Email <span class="operator">=</span> p2.Email <span class="keyword">AND</span> p1.Id <span class="operator">></span> p2.Id;</span><br></pre></td></tr></table></figure><h3 id="197-Rising-Temperature"><a href="#197-Rising-Temperature" class="headerlink" title="197.Rising Temperature"></a>197.<a href="https://leetcode.com/problems/rising-temperature/">Rising Temperature</a></h3><p>题目要求为查询所有当天温度比前一天高的日期的Id.<br>解题思路，首先用MySQL内置函数TO_DAYS()将日期转为天数，再使用错位比较（同180）找出温度大于前一天的日期，最后取交集。<br>查询语句如下：</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> w1.Id </span><br><span class="line"><span class="keyword">FROM</span> Weather w1 </span><br><span class="line"><span class="keyword">INNER</span> <span class="keyword">JOIN</span> Weather w2</span><br><span class="line"><span class="keyword">ON</span> TO_DAYS(w1.Date) <span class="operator">=</span> TO_DAYS(w2.Date) <span class="operator">+</span> <span class="number">1</span> </span><br><span class="line"><span class="keyword">AND</span> w1.Temperature <span class="operator">></span> w2.Temperature;</span><br></pre></td></tr></table></figure><h3 id="262-Trips-and-Users"><a href="#262-Trips-and-Users" class="headerlink" title="262.Trips and Users"></a>262.<a href="https://leetcode.com/problems/trips-and-users/">Trips and Users</a></h3><p>最后一道题，看起来就像压轴题……<br>题目要求为查询出Oct 1,2013至Oct 3,2013之间，非禁止用户的请求撤销率。<br>解题思路为根据要求链接两表，设置好过滤条件，再将结果计算后返回。</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> </span><br><span class="line"> t.Request_at <span class="keyword">AS</span> `<span class="keyword">Day</span>`, </span><br><span class="line"> ROUND(</span><br><span class="line"> <span class="built_in">SUM</span>(</span><br><span class="line"> IF(t.Status <span class="operator">=</span> <span class="string">'completed'</span>, <span class="number">0</span>, <span class="number">1</span>)</span><br><span class="line"> ) <span class="operator">/</span> <span class="built_in">COUNT</span>(<span class="operator">*</span>), <span class="number">2</span>) </span><br><span class="line"> <span class="keyword">AS</span> `Cancellation Rate` </span><br><span class="line"><span class="keyword">FROM</span> Trips t </span><br><span class="line"><span class="keyword">INNER</span> <span class="keyword">JOIN</span> Users u </span><br><span class="line"><span class="keyword">ON</span> t.Client_Id <span class="operator">=</span> u.Users_Id </span><br><span class="line"><span class="keyword">WHERE</span> t.Request_at </span><br><span class="line"><span class="keyword">BETWEEN</span> <span class="string">'2013-10-01'</span> <span class="keyword">AND</span> <span class="string">'2013-10-03'</span> </span><br><span class="line"><span class="keyword">AND</span> u.Banned <span class="operator">=</span> <span class="string">'No'</span> <span class="keyword">AND</span> u.Role <span class="operator">=</span> <span class="string">'client'</span> </span><br><span class="line"><span class="keyword">GROUP</span> <span class="keyword">BY</span> t.Request_at </span><br><span class="line"><span class="keyword">ORDER</span> <span class="keyword">BY</span> t.Request_at;</span><br></pre></td></tr></table></figure><p>通过之前的解题，理解这段查询语句的逻辑不难。这里仅把前面没提到过的内置函数做下笔记：</p><ul><li><code>ROUND()</code>为四舍五入函数，第二个参数为小数位保留的位数；</li><li><code>SUM()</code>顾名思义为求和函数；</li><li><code>IF()</code>,格式为IF(Condition,A,B)，当Condition为TRUE时，返回A；当Condition为FALSE时，返回B。</li></ul><h3 id="小结"><a href="#小结" class="headerlink" title="小结"></a>小结</h3><p>至此，Leetcode所有Database题目完成（撒花）。总的来看，这些题目锻炼了使用SQL的基本功，同时提供了思考问题的新思路，这在以后的工作中都很有帮助。同时在每页的Discuss中，查看其它网友的解法，能够进一步提高姿势水平。将这些解题的思路整理成两篇文章，也是一个不小的收获。</p>]]></content>
<summary type="html">这篇文章承接上一篇，将Leetcode.Database中的后6道题的解法及笔记记录下来。</summary>
<category term="MySQL" scheme="https://catinsides.github.io/tags/MySQL/"/>
</entry>
<entry>
<title>Leetcode.Database 题解笔记（上）</title>
<link href="https://catinsides.github.io/2016/12/26/Leetcode.Database-%E9%A2%98%E8%A7%A3%E7%AC%94%E8%AE%B0%EF%BC%88%E4%B8%8A%EF%BC%89/"/>
<id>https://catinsides.github.io/2016/12/26/Leetcode.Database-%E9%A2%98%E8%A7%A3%E7%AC%94%E8%AE%B0%EF%BC%88%E4%B8%8A%EF%BC%89/</id>
<published>2016-12-26T03:55:30.000Z</published>
<updated>2024-08-14T08:42:02.740Z</updated>
<content type="html"><![CDATA[<blockquote><p>Leetcode是一个在线编程网站，收录了算法，数据库等各种题目。同样有很多优秀的算法解法，能够有效的提高姿势水平。</p></blockquote><p>闲来无事在leetcode上刷了所有的database题目，一共13道题。题目内容不是简单的CRUD，而是可能很“常用”的查询问题。自我感觉，刷题的过程是一种查缺补漏的过程。除了能够学习新知识，还能够找到自己的不足。这里将每道题的解法及思路记录下来，方便以后查阅。<br>由于每道题的题目内容较多，仅写下解法与笔记，链接附在题目中。</p><h3 id="175-Combine-Two-Tables"><a href="#175-Combine-Two-Tables" class="headerlink" title="175.Combine Two Tables"></a>175.<a href="https://leetcode.com/problems/combine-two-tables/">Combine Two Tables</a></h3><p>题目要求为取出Person表中每一个人的FirstName，LastName，City，State信息，无论是否存在地址信息。<br>解题思路很简单，使用LEFT JOIN即可：</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> Person.FirstName,Person.LastName,Address.City,Address.State</span><br><span class="line"><span class="keyword">FROM</span> Person <span class="keyword">LEFT</span> <span class="keyword">JOIN</span> Address </span><br><span class="line"><span class="keyword">ON</span> Person.PersonId <span class="operator">=</span> Address.PersonId;</span><br></pre></td></tr></table></figure><p>这么写可能显得臃肿，可以简化一下：</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> p.FirstName,p.LastName,a.City,a.State</span><br><span class="line"><span class="keyword">FROM</span> Person p </span><br><span class="line"><span class="keyword">LEFT</span> <span class="keyword">JOIN</span> Address a</span><br><span class="line"><span class="keyword">USING</span> (PersonId);</span><br></pre></td></tr></table></figure><p>JOIN USING 简化 JOIN ON 的条件是</p><ul><li>查询必须等连接；</li><li>等连接的列必须同名。</li></ul><h3 id="176-Second-Highest-Salary"><a href="#176-Second-Highest-Salary" class="headerlink" title="176.Second Highest Salary"></a>176.<a href="https://leetcode.com/problems/second-highest-salary/">Second Highest Salary</a></h3><p>题目很好理解，查询Employee表中第二高的薪水值。<br>解题思路为使用MAX()函数：</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> <span class="built_in">max</span>(Salary)</span><br><span class="line"><span class="keyword">FROM</span> Employee</span><br><span class="line"><span class="keyword">WHERE</span> Salary <span class="operator"><</span> (<span class="keyword">SELECT</span> <span class="built_in">max</span>(Salary) <span class="keyword">FROM</span> Employee);</span><br></pre></td></tr></table></figure><p>提交完后，会报错，因为在第一行后面还要加上 <code>as SecondHighestSalary</code>，真是蛋疼的设定啊…</p><h3 id="177-Nth-Highest-Salary"><a href="#177-Nth-Highest-Salary" class="headerlink" title="177.Nth Highest Salary"></a>177.<a href="https://leetcode.com/problems/nth-highest-salary/">Nth Highest Salary</a></h3><p>题目要求为取出表内第N高的薪水值，如果没有则返回null.<br>这里要编写的为自定义函数，方法如下：</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">CREATE</span> <span class="keyword">FUNCTION</span> getNthHighestSalary(N <span class="type">INT</span>) <span class="keyword">RETURNS</span> <span class="type">INT</span></span><br><span class="line"><span class="keyword">BEGIN</span></span><br><span class="line"><span class="keyword">DECLARE</span> M <span class="type">INT</span>;</span><br><span class="line"><span class="keyword">SET</span> M <span class="operator">=</span> N<span class="number">-1</span>;</span><br><span class="line"> <span class="keyword">RETURN</span> (</span><br><span class="line"> <span class="keyword">SELECT</span> <span class="keyword">DISTINCT</span> Salary <span class="keyword">FROM</span> Employee <span class="keyword">ORDER</span> <span class="keyword">BY</span> Salary <span class="keyword">DESC</span> LIMIT M, <span class="number">1</span></span><br><span class="line"> );</span><br><span class="line"><span class="keyword">END</span></span><br></pre></td></tr></table></figure><p>简单记一下LIMIT的用法吧：</p><ul><li>LIMIT 5, 1 //从第6行取出1个</li><li>LIMIT 5, -1 //从第6行至last</li><li>LIMIT 5 // 等价于LIMIT 0, 5 前五行</li></ul><h3 id="178-Rank-Scores"><a href="#178-Rank-Scores" class="headerlink" title="178.Rank Scores"></a>178.<a href="https://leetcode.com/problems/rank-scores/">Rank Scores</a></h3><p>题目内容为按次序排名分数，分数相同的次序相同，分数之间的次序不能跳跃。<br>解题思路为新建临时表提取去重后的分数，再使用COUNT计数，再与原表中的分数进行笛卡尔积，最后再设置条件筛选：</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> s.Score, <span class="built_in">COUNT</span>(r.Score) <span class="keyword">AS</span> Rank</span><br><span class="line"><span class="keyword">FROM</span> Scores s</span><br><span class="line"> , (</span><br><span class="line"> <span class="keyword">SELECT</span> <span class="keyword">DISTINCT</span> Score</span><br><span class="line"> <span class="keyword">FROM</span> Scores</span><br><span class="line"> ) r</span><br><span class="line"><span class="keyword">WHERE</span> s.Score <span class="operator"><=</span> r.Score</span><br><span class="line"><span class="keyword">GROUP</span> <span class="keyword">BY</span> s.Id, s.Score</span><br><span class="line"><span class="keyword">ORDER</span> <span class="keyword">BY</span> s.Score <span class="keyword">DESC</span>;</span><br></pre></td></tr></table></figure><p>其中，GOURP BY为分组计数。</p><h3 id="180-Consecutive-Numbers"><a href="#180-Consecutive-Numbers" class="headerlink" title="180.Consecutive Numbers"></a>180.<a href="https://leetcode.com/problems/consecutive-numbers/">Consecutive Numbers</a></h3><p>题目内容为查询出至少出现3次的数字。<br>解题思路比较巧妙，使用JOIN和新建临时表匹配错位相等的值</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> <span class="keyword">DISTINCT</span> L1.Num <span class="keyword">AS</span> ConsecutiveNums</span><br><span class="line"><span class="keyword">FROM</span> Logs L1</span><br><span class="line"><span class="keyword">JOIN</span> Logs L2 <span class="keyword">ON</span> L1.Id <span class="operator">+</span> <span class="number">1</span> <span class="operator">=</span> L2.Id</span><br><span class="line"><span class="keyword">JOIN</span> Logs L3 <span class="keyword">ON</span> L1.Id <span class="operator">+</span> <span class="number">2</span> <span class="operator">=</span> L3.Id</span><br><span class="line"><span class="keyword">WHERE</span> L1.Num <span class="operator">=</span> L2.Num <span class="keyword">AND</span> L1.Num <span class="operator">=</span> L3.Num</span><br><span class="line"><span class="keyword">ORDER</span> <span class="keyword">BY</span> L1.Num</span><br></pre></td></tr></table></figure><p>这道题考验思路与技巧，没有什么额外的内容。</p><h3 id="181-Employees-Earning-More-Than-Their-Managers"><a href="#181-Employees-Earning-More-Than-Their-Managers" class="headerlink" title="181.Employees Earning More Than Their Managers"></a>181.<a href="https://leetcode.com/problems/employees-earning-more-than-their-managers/">Employees Earning More Than Their Managers</a></h3><p>题目内容为查找出比经理收入高的员工。<br>这道题属于Easy，没啥太大的难点，需要注意的是员工与经理在同一表内，通过Id关联，注意筛选条件即可。</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">SELECT</span> a.NAME <span class="keyword">AS</span> Employee </span><br><span class="line"><span class="keyword">FROM</span> Employee a, Employee b </span><br><span class="line"><span class="keyword">WHERE</span> a.ManagerId <span class="operator">=</span> b.Id <span class="keyword">AND</span> a.Salary <span class="operator">></span> b.Salary;</span><br></pre></td></tr></table></figure><h3 id="182-Duplicate-Emails"><a href="#182-Duplicate-Emails" class="headerlink" title="182.Duplicate Emails"></a>182.<a href="https://leetcode.com/problems/duplicate-emails/">Duplicate Emails</a></h3><p>题目内容为查找表中重复的emails.<br>这道题也很简单，使用GROUP BY和HAVING 即可。它的作用是代替<code>WHERE</code>与合计函数一起使用。</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">SELECT Email FROM Person GROUP BY Email HAVING COUNT(*) > 1;</span><br></pre></td></tr></table></figure><p>先把前7道题写到这里，后面6道题包含HARD难度，知识点比较多，单独开一篇吧。</p>]]></content>
<summary type="html">Leetcode是一个在线编程网站，收录了算法，数据库等各种题目。同样有很多优秀的算法解法，能够有效的提高姿势水平。</summary>
<category term="MySQL" scheme="https://catinsides.github.io/tags/MySQL/"/>
</entry>
<entry>
<title>SCRAPY 简单入门</title>
<link href="https://catinsides.github.io/2016/12/19/SCRAPY-%E7%AE%80%E5%8D%95%E5%85%A5%E9%97%A8/"/>
<id>https://catinsides.github.io/2016/12/19/SCRAPY-%E7%AE%80%E5%8D%95%E5%85%A5%E9%97%A8/</id>
<published>2016-12-19T10:37:56.000Z</published>
<updated>2024-08-14T08:42:02.741Z</updated>
<content type="html"><![CDATA[<blockquote><p>SCRAPY是由Python开发的，为爬取网站数据，提取结构性数据而编写的应用框架。可以应用在包括数据挖掘，信息处理或存储历史数据等一系列的程序中。</p></blockquote><p>下面通过一个小例子，爬取RS05网站的电影链接，熟悉该框架的基本功能。<br>闲话少叙，下面开始:)</p><h3 id="安装SCRAPY"><a href="#安装SCRAPY" class="headerlink" title="安装SCRAPY"></a>安装SCRAPY</h3><p>相信有一半的同学，学习SCRAPY从安装到放弃。<br>我也是折腾了好久，从baidu到google，再到stackoverflow，才完完整整的安装到了电脑上。高兴的我，又在树莓派Raspbian系统上安装了一遍。为了节省大家的时间，这里把Linux和Windows的安装方法都列举出来，方便以后查询。<br>Python版本为2.7</p><h4 id="Windows平台："><a href="#Windows平台：" class="headerlink" title="Windows平台："></a>Windows平台：</h4><p>首先到<a href="https://www.microsoft.com/en-us/download/details.aspx?id=44266">这里</a>下载VC For Python27<br>然后按照下列顺序安装前置包</p><ul><li>zope.interface</li><li>pyopenssl</li><li>twisted</li><li>lxml</li><li>scrapy</li></ul><p>不能使用pip下载的，直接google安装包好了。</p><h4 id="Linux平台："><a href="#Linux平台：" class="headerlink" title="Linux平台："></a>Linux平台：</h4><p>准确的说应该是Raspbian平台，因为我用的树莓派是XD。相比win平台就简单多了，</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">apt-get install libxml2-dev libxslt1-dev python-dev</span><br><span class="line">apt-get install python-lxml</span><br><span class="line"><span class="keyword">if</span> error:</span><br><span class="line">pip install pyasn1 --upgrade</span><br></pre></td></tr></table></figure><p>好的，大功告成！enjoy！</p><h3 id="选择网站"><a href="#选择网站" class="headerlink" title="选择网站"></a>选择网站</h3><p>我真是太懒了，懒到不想点开浏览器去找电影下载。为了节省十几秒找电影的时间，花去了几个小时研究爬虫框架自动爬取链接。想想还是挺值的！<br>这里我选择RS05为目标网站，提取每个电影页面的迅雷url并写入文件，然后直接复制到迅雷里就可以下载了，是不是很简单很开心？</p><h3 id="新建项目"><a href="#新建项目" class="headerlink" title="新建项目"></a>新建项目</h3><p>好的，终于要开始了。<br>CD到项目文件夹，执行命令：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">scrapy startproject Rs05</span><br></pre></td></tr></table></figure><p>随后项目文件夹内变生成了一系列文件，其结构如下：</p><blockquote><p>Rs05/<br> scrapy.cfg<br> Rs05/<br> <code>__init__.py</code><br> items.py<br> piplines.py<br> settings.py<br> spiders/<br> <code>__init__.py</code><br> …</p></blockquote><p>简单介绍一下各个文件：<br><code>scrapy.cfg</code> 是项目的配置文件；<br><code>__init__.py</code> 这个文件不用管；<br><code>items.py</code> 文件里编写爬取内容的Field；<br><code>piplines.py</code> 文件里编写爬取后过滤内容的规则或者其他操作，如写入文件；<br><code>settings.py</code> 是项目的设置文件；<br><code>spiders</code> 内放置爬虫文件</p><h3 id="设置规则"><a href="#设置规则" class="headerlink" title="设置规则"></a>设置规则</h3><p>首先，需要定义items，可以理解为放置爬取内容的地方，与字典类似。<br>编写方法非常简单，</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> scrapy</span><br><span class="line"><span class="keyword">class</span> <span class="title class_">Rs05Item</span>(scrapy.Item):</span><br><span class="line"> <span class="comment">#name = scrapy.Field()</span></span><br><span class="line"> title = scrapy.Field()</span><br><span class="line"> url = scrapy.Field()</span><br></pre></td></tr></table></figure><p>name根据你的需要随便起名字，数量由爬取种类的多少决定。<br>然后CD到spider文件，新建爬虫文件rs05_spider.py<br>打开文件编写</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#coding:utf-8</span></span><br><span class="line"><span class="keyword">import</span> re</span><br><span class="line"><span class="keyword">import</span> scrapy</span><br><span class="line"><span class="keyword">from</span> scrapy.contrib.spiders <span class="keyword">import</span> CrawlSpider, Rule</span><br><span class="line"><span class="keyword">from</span> scrapy.contrib.linkextractors <span class="keyword">import</span> LinkExtractor</span><br><span class="line"><span class="keyword">from</span> rs05.items <span class="keyword">import</span> Rs05Item</span><br><span class="line"><span class="keyword">class</span> <span class="title class_">Rs05Spider</span>(<span class="title class_ inherited__">CrawlSpider</span>):</span><br><span class="line"> name = <span class="string">'rs05'</span></span><br><span class="line"> start_urls = [<span class="string">'http://www.rs05.com/'</span>]</span><br><span class="line"> rules = (</span><br><span class="line"> Rule(LinkExtractor(allow=(<span class="string">'[a-z0-9]+\.html'</span>)),callback=<span class="string">'parse_item'</span>),</span><br><span class="line"> )</span><br><span class="line"> <span class="keyword">def</span> <span class="title function_">parse_item</span>(<span class="params">self,response</span>):</span><br><span class="line"> items = []</span><br><span class="line"> item = Rs05Item()</span><br><span class="line"> <span class="keyword">for</span> sel <span class="keyword">in</span> response.xpath(<span class="string">'/html/body/div[4]/div[1]/div[1]'</span>):</span><br><span class="line"> title = sel.xpath(<span class="string">'h1/text()'</span>).extract()</span><br><span class="line"> item[<span class="string">'title'</span>] = title <span class="comment">#t.encode('utf-8') for t in title</span></span><br><span class="line"> pattern = re.<span class="built_in">compile</span>(<span class="string">'href="(.*?)"'</span>,re.S)</span><br><span class="line"> url = re.findall(pattern, sel.extract())</span><br><span class="line"> item[<span class="string">'url'</span>] = url</span><br><span class="line"> items.append(item)</span><br><span class="line"> <span class="keyword">return</span> items</span><br></pre></td></tr></table></figure><p>根据代码可以看出，爬虫使用了scrapy内置的CrawlSpider，用于跟进链接继续爬取。而linkextractors用于设置跟进链接的规则，使用正则匹配。<code>name</code>为爬虫的名称，该名称唯一。<code>start_urls</code>为开始爬取的链接。需要注意的是，当要设置<code>allowed_domains</code>时，<code>start_urls</code>与<code>allowed_domains</code>不能相同，否则会出现错误。<code>parse_item</code>函数用于解析response的内容，并将结果保存到items中。上述代码使用xpath与re提取内容，当然也可以使用其他方法，比如beautifulsoup。无论用什么方法，最终将需要提取的内容放到items中即可。</p><h3 id="写入文件"><a href="#写入文件" class="headerlink" title="写入文件"></a>写入文件</h3><p>爬取内容后，如何输出呢？<br>此时打开pipelines.py，输入以下代码：</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> json</span><br><span class="line"><span class="keyword">import</span> codecs</span><br><span class="line"><span class="keyword">from</span> collections <span class="keyword">import</span> OrderedDict</span><br><span class="line"><span class="keyword">class</span> <span class="title class_">Rs05Pipeline</span>(<span class="title class_ inherited__">object</span>):</span><br><span class="line"> <span class="keyword">def</span> <span class="title function_">__init__</span>(<span class="params">self</span>):</span><br><span class="line"> self.file = codecs.<span class="built_in">open</span>(<span class="string">'rs05.json'</span>,<span class="string">'wb'</span>,encoding=<span class="string">'utf-8'</span>)</span><br><span class="line"> <span class="keyword">def</span> <span class="title function_">process_item</span>(<span class="params">self, item, spider</span>):</span><br><span class="line"> line = json.dumps(OrderedDict(item)) + <span class="string">'\n'</span></span><br><span class="line"> self.file.write(line.decode(<span class="string">"unicode_escape"</span>))</span><br><span class="line"> <span class="keyword">return</span> item</span><br><span class="line"> <span class="keyword">def</span> <span class="title function_">close_spider</span>(<span class="params">self, spider</span>):</span><br><span class="line"> self.file.close()</span><br></pre></td></tr></table></figure><p>其中,<code>__init__()</code>和<code>close_spider()</code>相当于构造、析构函数，写不写都可以。但是process_item是一定要写的。这里用于处理爬取内容，丢弃或者保存。从上面的代码可以看出，爬取内容保存到了rs05.json文件中。<br>还没有完事，打开settings.py找到</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">ITEM_PIPELINES = {</span><br><span class="line"> <span class="string">'rs06.pipelines.Rs06Pipeline'</span>: <span class="number">300</span>,</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>取消注释，并将名字改为爬虫的名字，后面的数字为优先级，如果你有多个pipelines，可以为它们设定执行顺序。</p><h3 id="开始爬取"><a href="#开始爬取" class="headerlink" title="开始爬取"></a>开始爬取</h3><p>好了，一切都设置好之后，CD到项目根目录可以开始爬取了。<br>执行下列命令：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">scrapy crawl Rs05</span><br></pre></td></tr></table></figure><p>一大串字母和数字闪过之后，在根目录生成了rs05.json, 打开文件看看是否提取到了想要的内容。我相信多数情况下，都没有一次成功的时候吧……<br>下面介绍两种调试的方法：</p><ul><li>1.检查spider文件的过滤规则，是否提取到内容。如果使用xpath，可以通过xpath插件，或者chrome shift + i 查看xpath路径；</li><li>2.输入命令 <code>scrapy shell "http://xxx"</code> 然后输入 <code>response.selector.xpath('your rules')</code>查看内容，验证xpath路径是否正确。</li></ul><h3 id="小结"><a href="#小结" class="headerlink" title="小结"></a>小结</h3><p>SCRAPY框架适用于中型大型爬虫项目，使我们专注于内容的提取，而不是爬虫的构建。小型项目可以使用urllib,repuests库等等。<br>简单来讲，网络爬虫就是获取网页内容后提取有价值信息的一个技术。我们设置好过滤内容规则后，让电脑自动提取，大大加强了我们提取信息的效率。当然，这一过程中，也会遇到一些困难，如网页改版，封禁IP等等。此时就需要将爬虫升级为机器学习或者分布式爬虫，这里需要更多更深的知识。</p>]]></content>
<summary type="html">SCRAPY是由Python开发的，为爬取网站数据，提取结构性数据而编写的应用框架。可以应用在包括数据挖掘，信息处理或存储历史数据等一系列的程序中。</summary>
<category term="Hello world" scheme="https://catinsides.github.io/tags/Hello-world/"/>
<category term="Python" scheme="https://catinsides.github.io/tags/Python/"/>
</entry>
<entry>
<title>CodeIgniter 简单入门</title>
<link href="https://catinsides.github.io/2016/11/26/CodeIgniter-%E7%AE%80%E5%8D%95%E5%85%A5%E9%97%A8/"/>
<id>https://catinsides.github.io/2016/11/26/CodeIgniter-%E7%AE%80%E5%8D%95%E5%85%A5%E9%97%A8/</id>
<published>2016-11-26T09:42:56.000Z</published>
<updated>2024-08-14T08:42:02.737Z</updated>
<content type="html"><![CDATA[<blockquote><p>CodeIgniter 是一套给 PHP 网站开发者使用的应用程序开发框架和工具包。 它的目标是让你能够更快速的开发，它提供了日常任务中所需的大量类库， 以及简单的接口和逻辑结构。通过减少代码量，CodeIgniter 让你更加专注于你的创造性工作。</p></blockquote><p>本文介绍该框架（简称CI）的基本功能，方便自己查阅和回顾。<br>闲话少叙，下面开始:)</p><h3 id="环境"><a href="#环境" class="headerlink" title="环境"></a>环境</h3><p>测试环境建议使用wamp或lamp，下载后安装即可。<br>从CI官网下载压缩包，解压到wamp(lamp)目录下的www文件夹内，<br>在浏览器地址栏输入 <a href="http://localhost/">http://localhost</a> ，即可看到欢迎界面。</p><h3 id="路由"><a href="#路由" class="headerlink" title="路由"></a>路由</h3><p>路由配置文件routes.php在框架根目录下application/config文件夹内，打开后看到一堆注释和保留语句。注释部分算是一部分教程，方便理解框架内容。<br>保留语句是路由名称数组，左面是匹配规则，右面是对应的控制器类。<br>仅保留这一条规则，其他规则注释掉。</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="variable">$route</span>[<span class="string">'default_controller'</span>] = <span class="string">'welcome'</span>;</span><br></pre></td></tr></table></figure><p>这行代码的含义是，当打开 <a href="http://localhost/">http://localhost</a> 时，默认使用Welcome控制器。<br>噫，这里怎么只有控制器名没有方法名呢？<br>这时打开application/controller下的Welcome.php，会发现里面有一个index()方法，通过上面的注释可以了解到，访问<br><a href="http://localhost/index.php/welcome">http://localhost/index.php/welcome</a><br>或<br><a href="http://localhost/index.php/welcome/index">http://localhost/index.php/welcome/index</a><br>都会匹配到index()方法。<br>即当url中没有指明方法名称时，CI会自动寻找当前控制器类下的index()方法执行。</p><p>好了，上面都是官方的范例，现在轮到自己动手了。<br>情景设定为，从数据库中获取所有的优惠码显示在页面中。<br>新建路由规则，</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="variable">$route</span>[<span class="string">'codes'</span>] = <span class="string">'main/get'</span>;</span><br></pre></td></tr></table></figure><p>访问 <a href="http://localhost/index.php/codes">http://localhost/index.php/codes</a> 时，即可使用Main控制类下的get()方法。</p><p>路由规则也可以使用正则限定参数，例</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="variable">$route</span>[<span class="string">'codes/([a-z]+)/(\d+)'</span>] = <span class="string">'main/$1/$2'</span>;</span><br></pre></td></tr></table></figure><p>访问 <a href="http://localhost/index.php/codes/get/1">http://localhost/index.php/codes/get/1</a> 时，会定向到main类下的get()方法，多余的url参数会传入到函数中。</p><h3 id="控制器"><a href="#控制器" class="headerlink" title="控制器"></a>控制器</h3><p>在目录application/controller下，新建文件Main.php。<br>控制器类文件名可根据编程规范命名，这里为了方便仅采用了首字母大写的规则。<br>打开文件，键入代码</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"><?php</span></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Main</span> <span class="keyword">extends</span> <span class="title">CI_Controller</span> </span>{</span><br><span class="line"> }</span><br></pre></td></tr></table></figure><p>因为要从数据库中获取信息，所以可以把读取模型的代码写入到构造方法中，减少后面的代码量。</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">__construct</span>(<span class="params"></span>)</span>{</span><br><span class="line"> <span class="built_in">parent</span>::<span class="title function_ invoke__">__construct</span>();</span><br><span class="line"> <span class="variable language_">$this</span>->load-><span class="title function_ invoke__">model</span>(<span class="string">'coupon_model'</span>);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure><p>在get()方法中，直接使用模型类中的方法即可。<br>同时，把将要渲染模板的代码写入方法中，</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">get</span>(<span class="params"></span>)</span>{</span><br><span class="line"> <span class="variable">$data</span>[<span class="string">'codes'</span>] = <span class="variable language_">$this</span>->coupon-><span class="title function_ invoke__">get_codes</span>();</span><br><span class="line"> <span class="variable language_">$this</span>->load-><span class="title function_ invoke__">view</span>(<span class="string">'templates/header'</span>);</span><br><span class="line"> <span class="variable language_">$this</span>->load-><span class="title function_ invoke__">view</span>(<span class="string">'pages/content'</span>, <span class="variable">$data</span>);</span><br><span class="line"> <span class="variable language_">$this</span>->load-><span class="title function_ invoke__">view</span>(<span class="string">'templates/footer'</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>上面的view()方法会在<b>模板</b>章节介绍。</p><h3 id="模型"><a href="#模型" class="headerlink" title="模型"></a>模型</h3><p>数据库配置文件在目录application/config下，<br>打开database.php可以看相关配置参数，根据实际情况填写即可。<br>配置完成后，在目录application/models下，新建模型类文件Coupon_model.php,<br>打开文件输入以下代码即可完成配置，</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta"><?php</span></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Coupon_model</span> <span class="keyword">extends</span> <span class="title">CI_Model</span></span>{</span><br><span class="line"> <span class="function"><span class="keyword">function</span> <span class="title">__construct</span>(<span class="params"></span>)</span>{</span><br><span class="line"> <span class="variable language_">$this</span>->load-><span class="title function_ invoke__">database</span>();</span><br><span class="line"> }</span><br><span class="line"> <span class="function"><span class="keyword">function</span> <span class="title">get_codes</span>(<span class="params"></span>)</span>{</span><br><span class="line"> <span class="variable">$query</span> = <span class="variable language_">$this</span>->db-><span class="title function_ invoke__">get</span>(<span class="string">'test'</span>);</span><br><span class="line"> <span class="keyword">return</span> <span class="variable">$query</span>-><span class="title function_ invoke__">result_array</span>();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>返回数据格式为数组。<br>这里用到的数据库信息与THINKPHP快速入门中一致。<br>这样，框架的模型配置已经完成了。</p><h3 id="模板"><a href="#模板" class="headerlink" title="模板"></a>模板</h3><p>这里介绍第三节提到的渲染模板。根据代码可以看出view()方法接收了两个参数，其中一个是模板文件的路径，另一个则是传入的变量。<br>模板文件在目录application/views下，其中有官方模板可以借用。<br>新建两个文件夹pages和templates，与代码相对应。<br>顾名思义，templates是模板，可以将html通用代码分为header.html和footer.html两个文件，如<code><body></code>标签以上的部分写到header内，<code></body></code>以下的部分写到footer内。引用时不需要写后缀名。<br>在pages文件夹内创建content.html，输入代码</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><h2>这里是优惠码:</h2></span><br><span class="line"><ul></span><br><span class="line"><span class="meta"><?php</span> <span class="keyword">foreach</span> (<span class="variable">$data</span> <span class="keyword">as</span> <span class="variable">$date_item</span>):<span class="meta">?></span></span><br><span class="line"> <li><span class="meta"><?php</span> <span class="keyword">echo</span> <span class="variable">$date_item</span>[<span class="string">'id'</span>].<span class="string">":"</span>.<span class="variable">$date_item</span>[<span class="string">'code'</span>]; <span class="meta">?></span></li></span><br><span class="line"><span class="meta"><?php</span> <span class="keyword">endforeach</span>; <span class="meta">?></span></span><br><span class="line"></ul></span><br></pre></td></tr></table></figure><p>这样的代码很是让人头疼。CI提供了模板解析类，使用更优美的模板语言渲染模板。当然这部分内容不在本文的范围内，具体内容可以参考官方文档。<br>当模板渲染时，会按照view()的顺序依次渲染。</p><p>现在，在浏览器地址栏输入 <a href="http://localhost/index.php/codes">http://localhost/index.php/codes</a> ，会看到从数据库中获取的优惠码信息。</p><p>0:SUFFLzS<br>1:hZxakCl<br>2:xD2gz0N<br>3:6WwLt7U<br>4:xLSxY1n<br>5:bn422Ka<br>6:55BaQ0C<br>7:Ui1Jc8P</p><h3 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h3><p> CodeIgniter的开发遵循了MVC的设计模式，这种将逻辑层与表现层分离的软件方法，让开发人员更专注于自己所属的工作，同时代码的维护也方便了很多。<br> 个人非常喜欢CI框架，不仅是其简单自由高效的特性，更因为其语法风格。开发时配置明确，思路清晰，总之写起来很爽。</p>]]></content>
<summary type="html">CodeIgniter 是一套给 PHP 网站开发者使用的应用程序开发框架和工具包。它的目标是让你能够更快速的开发，它提供了日常任务中所需的大量类库， 以及简单的接口和逻辑结构。通过减少代码量，CodeIgniter 让你更加专注于你的创造性工作。</summary>
<category term="Hello world" scheme="https://catinsides.github.io/tags/Hello-world/"/>
<category term="PHP" scheme="https://catinsides.github.io/tags/PHP/"/>
</entry>
<entry>
<title>Threejs读取STL文件</title>
<link href="https://catinsides.github.io/2016/11/26/Threejs%E8%AF%BB%E5%8F%96STL%E6%96%87%E4%BB%B6/"/>
<id>https://catinsides.github.io/2016/11/26/Threejs%E8%AF%BB%E5%8F%96STL%E6%96%87%E4%BB%B6/</id>
<published>2016-11-26T09:42:46.000Z</published>
<updated>2024-08-14T08:42:02.742Z</updated>
<content type="html"><![CDATA[<blockquote><p>Threejs是使用JavaScript编写的，运行在浏览器中的3D引擎。</p></blockquote><p> 当初想在我的小项目中添加三维模型展示的功能，于是找到了这个js编写的第三方库。它可以方便的读取图形文件加载到浏览器中，并且提供各式小工具完成交互功能。<br> 我在工作中使用的三维软件是Solidworks，可以方便的导出STL格式的文件，这也是通常在3D打印中使用的格式。百度上搜到的threejs教程比较丰富，然而偏偏没有读取STL文件的方法。官方文档中提供了各种Loaders用于加载各种格式的图形文件，然而偏偏也没有读取STL文件的方法……<br> 最终神奇的在官方提供的代码包中(<a href="https://github.com/mrdoob/three.js" title="github">github</a>),找到了示例代码。分析代码，结合官方文档生肉，功夫不负有心人，总算是把生肉炒成了熟肉。<br> Talk is cheap,show me the code.</p><h3 id="添加画布"><a href="#添加画布" class="headerlink" title="添加画布"></a>添加画布</h3><p>其实就是在<code><body></code>标签内添加一个<code><div></code>,如下</p><figure class="highlight html"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag"><<span class="name">div</span> <span class="attr">id</span>=<span class="string">"output"</span>></span><span class="tag"></<span class="name">div</span>></span></span><br></pre></td></tr></table></figure><p>这个区域就是即将插入三维模型的地方，添加id属性方便获取。</p><h3 id="Threejs"><a href="#Threejs" class="headerlink" title="Threejs"></a>Threejs</h3><p>本代码中的功能需要引入官方包中的three.min.js, STLLoader.js和OrbitControls.js.</p><h4 id="场景"><a href="#场景" class="headerlink" title="场景"></a>场景</h4><p>初始化场景类</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> scene = <span class="keyword">new</span> <span class="variable constant_">THREE</span>.<span class="title class_">Scene</span>();</span><br></pre></td></tr></table></figure><p>为了便于观察，在场景中放置一个三坐标</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> axes = <span class="keyword">new</span> <span class="variable constant_">THREE</span>.<span class="title class_">AxisHelper</span>(<span class="number">20</span>);</span><br><span class="line">scene.<span class="title function_">add</span>(axes);</span><br></pre></td></tr></table></figure><p>需要在场景添加其他物件时，使用scene.add()即可，就是这么简单~</p><h4 id="照相机"><a href="#照相机" class="headerlink" title="照相机"></a>照相机</h4><p>这里的照相机可不是生活中的指示代词，而是图形学中一种抽象概念。它定义了三维空间到二维屏幕的投影方式，照相机又分为正交投影照相机和透视投影照相机。本代码中使用的是透视投影照相机，至于为什么使用这个方式，有兴趣的同学可以wiki两种照相机的区别。<br>初始化照相机</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> camera = <span class="keyword">new</span> <span class="variable constant_">THREE</span>.<span class="title class_">PerspectiveCamera</span>(<span class="number">45</span>, <span class="variable language_">window</span>.<span class="property">innerWidth</span> / <span class="variable language_">window</span>.<span class="property">innerHeight</span>, <span class="number">0.1</span>, <span class="number">1000</span>);</span><br><span class="line">camera.<span class="property">position</span>.<span class="property">x</span> = <span class="number">150</span>;</span><br><span class="line">camera.<span class="property">position</span>.<span class="property">y</span> = <span class="number">50</span>;</span><br><span class="line">camera.<span class="property">position</span>.<span class="property">z</span> = <span class="number">150</span>;</span><br><span class="line">camera.<span class="title function_">lookAt</span>(scene.<span class="property">position</span>);</span><br></pre></td></tr></table></figure><p>第一部分，PerspectiveCamera()接收四个参数：</p><ul><li>45，为照相机竖直方向张角（角度制）；</li><li>window.innerWidth / window.innerHeight,照相机水平与竖直方向长度的比值，取三维模型渲染区域的横纵比例；</li><li>0.1, 1000，最近点和最远点。</li></ul><p>这些参数决定模型投影的形状与大小，一般情况下默认即可。</p><p>第二部分，设定照相机的位置，这些数值可根据模型大小进行调整。<br>第三部分，使照相机指向原点。</p><h4 id="读取与渲染"><a href="#读取与渲染" class="headerlink" title="读取与渲染"></a>读取与渲染</h4><p>初始化类</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> webGLRenderer = <span class="keyword">new</span> <span class="variable constant_">THREE</span>.<span class="title class_">WebGLRenderer</span>();</span><br></pre></td></tr></table></figure><p>设定背景颜色与范围，并将渲染结果插入到页面中</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">webGLRenderer.<span class="title function_">setClearColor</span>(<span class="number">0xEEEEEE</span>);</span><br><span class="line">webGLRenderer.<span class="title function_">setSize</span>(<span class="variable language_">window</span>.<span class="property">innerWidth</span>, <span class="variable language_">window</span>.<span class="property">innerHeight</span>);</span><br><span class="line"><span class="variable language_">document</span>.<span class="title function_">getElementById</span>(<span class="string">"output"</span>).<span class="title function_">appendChild</span>(webGLRenderer.<span class="property">domElement</span>);</span><br></pre></td></tr></table></figure><p>初始化类，并读取模型文件</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> loader = <span class="keyword">new</span> <span class="variable constant_">THREE</span>.<span class="title class_">STLLoader</span>();</span><br><span class="line">loader.<span class="title function_">load</span>(<span class="string">"sample.stl"</span>, <span class="keyword">function</span> (<span class="params">geometry</span>) {</span><br><span class="line"> <span class="keyword">var</span> mat = <span class="keyword">new</span> <span class="variable constant_">THREE</span>.<span class="title class_">MeshNormalMaterial</span>();</span><br><span class="line"> mesh = <span class="keyword">new</span> <span class="variable constant_">THREE</span>.<span class="title class_">Mesh</span>(geometry, mat);</span><br><span class="line"> mesh.<span class="property">scale</span>.<span class="title function_">set</span>(<span class="number">0.3</span>, <span class="number">0.3</span>, <span class="number">0.3</span>);</span><br><span class="line"> scene.<span class="title function_">add</span>(mesh);</span><br><span class="line">});</span><br></pre></td></tr></table></figure><p>在匿名函数中，使用了内置的MeshNormalMaterial()材质，颜色分明便于观察。设定材质参数，添加到场景中。</p><h4 id="交互"><a href="#交互" class="headerlink" title="交互"></a>交互</h4><p>既然是浏览三维模型，所以模型必须能够随心所欲的从各个角度观看。<br>所以浏览器中的模型必须能够使用鼠标转动。官方包中提供了这一功能。<br>首先，初始化类</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">controls = <span class="keyword">new</span> <span class="variable constant_">THREE</span>.<span class="title class_">OrbitControls</span>( camera, webGLRenderer.<span class="property">domElement</span> );</span><br><span class="line">controls.<span class="property">enableDamping</span> = <span class="literal">true</span>;</span><br><span class="line">controls.<span class="property">dampingFactor</span> = <span class="number">0.25</span>;</span><br><span class="line">controls.<span class="property">enableZoom</span> = <span class="literal">true</span>;</span><br></pre></td></tr></table></figure><p>其实每转动一次，浏览器都在重新绘制模型，所以参数中引入了照相机与渲染类。<br>下面几个语句是使能拖动和缩放，默认即可。</p><p><b>这里是重点！</b><br>以上代码都完成后，我发现模型在浏览器中并不能动！<br>为什么不动呢？这曾经是困扰了我很长时间的问题。<br>经过各种的google和stackoverflow后，我才找到了解决方法！<br>那就是requestAnimationFrame ()，通过递归调用更新画面，使画面动起来。<br>直接上代码</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="title function_">render</span>();</span><br><span class="line"><span class="keyword">function</span> <span class="title function_">render</span>(<span class="params"></span>) {</span><br><span class="line"> <span class="title function_">requestAnimationFrame</span>(render);</span><br><span class="line"> controls.<span class="title function_">update</span>();</span><br><span class="line"> webGLRenderer.<span class="title function_">render</span>(scene, camera);</span><br><span class="line"> }</span><br></pre></td></tr></table></figure><p>代码完成，全部代码以及预览效果可以到我的Repositories中查看。</p><h3 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h3><p> 通过以上配置就可以将STL图形插入到网页中去了，并且可以使用鼠标拖动和缩放。<br>代码中的<code><div id="output"></code>可以任意大小，任意位置。STL文件在网上能够搜到各种各样的模型，替换代码中的文件名即可。<br> 如果想自己制作STL文件，推荐软件SketchUp.<br> 在使用chrome浏览器时，会提示跨域错误，不能加载模型。此时右击浏览器图标->属性，在属性目标后（引号外）添加 –allow-file-access-from-files.<br> FireFox无此问题。</p>]]></content>
<summary type="html">Threejs是使用JavaScript编写的，运行在浏览器中的3D引擎。</summary>
<category term="Hello world" scheme="https://catinsides.github.io/tags/Hello-world/"/>
<category term="Javascript" scheme="https://catinsides.github.io/tags/Javascript/"/>
</entry>
<entry>
<title>THINKPHP 简单入门</title>
<link href="https://catinsides.github.io/2016/11/25/THINKPHP-%E7%AE%80%E5%8D%95%E5%85%A5%E9%97%A8/"/>
<id>https://catinsides.github.io/2016/11/25/THINKPHP-%E7%AE%80%E5%8D%95%E5%85%A5%E9%97%A8/</id>
<published>2016-11-25T09:45:47.000Z</published>
<updated>2024-08-14T08:42:02.742Z</updated>
<content type="html"><![CDATA[<blockquote><p>THINKPHP是一个快速简单的轻量级国产PHP开发框架。</p></blockquote><p>本文介绍了其基础的功能，方便自己查阅和回顾。<br>闲话少叙，下面开始:)</p><h3 id="一、环境"><a href="#一、环境" class="headerlink" title="一、环境"></a>一、环境</h3><p>建议使用wamp、lamp搭建测试环境，下载安装即可。<br>从官网下载THINKPHP 3.2版本的压缩包，解压后放到wamp(lamp)下的www文件夹内。<br>在浏览器地址栏输入<a href="http://localhost/">http://localhost</a>, 即可看到欢迎界面。</p><h3 id="二、路由"><a href="#二、路由" class="headerlink" title="二、路由"></a>二、路由</h3><p>THINKPHP采用模块化设计，每个模块文件下都有相应的配置文件。<br>本文只配置一个模块Home，为了后面方便，仅需在Application/Common/Conf/config.php<br>配置即可。即此后提到的配置文件语句，均写到此文件内。<br>打开config.php, 保留语句为</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">return</span> <span class="keyword">array</span>();</span><br></pre></td></tr></table></figure><p>在括号内写入，</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="string">'URL_ROUTER_ON'</span> => <span class="literal">true</span>, <span class="comment">//开启路由</span></span><br><span class="line"><span class="string">'URL_ROUTE_RULES'</span> => <span class="keyword">array</span>( <span class="comment">//CURD操作</span></span><br><span class="line"> <span class="string">'create'</span> => <span class="string">'Home/Index/cre'</span>,</span><br><span class="line"> <span class="string">'update'</span> => <span class="string">'Home/Index/upd'</span>,</span><br><span class="line"> <span class="string">'retrieve'</span> => <span class="string">'Home/Index/ret'</span>,</span><br><span class="line"> <span class="string">'delete'</span> => <span class="string">'Home/Index/del'</span>,</span><br><span class="line"> ), </span><br></pre></td></tr></table></figure><p>此处路由规则的含义是，当在浏览器地址栏输入<br><a href="http://localhost/index.php/create">http://localhost/index.php/create</a><br>时，相当于访问<br><a href="http://localhost/index.php/Home/Index/cre">http://localhost/index.php/Home/Index/cre</a>,<br>即Home模块下Index控制类下的cre方法，以此类推。<br>当路由中包含参数时，在参数前添加’:’，匹配时$_GET[]会获取相应的参数，例</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">'create/:id\d/:code\d' => 'Home/Index/cre',</span><br></pre></td></tr></table></figure><p>\d为使用正则限制路由参数。</p><h3 id="三、控制器"><a href="#三、控制器" class="headerlink" title="三、控制器"></a>三、控制器</h3><p>控制器类文件命名的规则是首字母大写，驼峰命名法，并以.class.php结尾。<br>在路径Application/Home/Controller/下，新建文件重命名为IndexController.class.php<br>打开文件添加代码，</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">namespace</span> <span class="title class_">Home</span>\<span class="title class_">Controller</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Think</span>\<span class="title">Controller</span>;</span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">IndexController</span> <span class="keyword">extends</span> <span class="title">Controller</span> </span>{</span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">ret</span>(<span class="params"></span>)</span>{</span><br><span class="line"> <span class="variable">$Test</span> = <span class="title function_ invoke__">D</span>(<span class="string">'test'</span>); <span class="comment">//使用内置D方法初始化模型类Test</span></span><br><span class="line"> <span class="variable">$temp</span> = <span class="variable">$Test</span>-><span class="title function_ invoke__">get_coupon</span>(); <span class="comment">//使用模型类中的get_coupon()方法</span></span><br><span class="line"> <span class="variable language_">$this</span> -> <span class="title function_ invoke__">assign</span>(<span class="string">'temp'</span>,<span class="variable">$temp</span>); <span class="comment">//模板赋值</span></span><br><span class="line"> <span class="variable language_">$this</span> -> <span class="title function_ invoke__">display</span>(<span class="string">'main'</span>); <span class="comment">//模板渲染</span></span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">upd</span>(<span class="params"></span>)</span>{</span><br><span class="line"> <span class="variable">$data</span>[<span class="string">'id'</span>] = <span class="number">99</span>;</span><br><span class="line"> <span class="variable">$data</span>[<span class="string">'code'</span>] = <span class="number">123456789</span>;</span><br><span class="line"> <span class="variable">$Test</span> = <span class="title function_ invoke__">M</span>(<span class="string">'test'</span>); <span class="comment">//内置M方法</span></span><br><span class="line"> <span class="variable">$Test</span> -> <span class="title function_ invoke__">add</span>(<span class="variable">$data</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">cre</span>(<span class="params"></span>)</span>{</span><br><span class="line"> <span class="variable">$_GET</span>[<span class="string">'id'</span>] = <span class="variable">$data</span>[<span class="string">'id'</span>]; <span class="comment">//获取url中的参数</span></span><br><span class="line"> <span class="variable">$_GET</span>[<span class="string">'code'</span>] = <span class="variable">$data</span>[<span class="string">'code'</span>];</span><br><span class="line"> <span class="variable">$Test</span> = <span class="title function_ invoke__">M</span>(<span class="string">'test'</span>);</span><br><span class="line"> <span class="variable">$Test</span> -> <span class="title function_ invoke__">where</span>(<span class="string">'id=2'</span>) -> <span class="title function_ invoke__">save</span>(<span class="variable">$data</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">del</span>(<span class="params"></span>)</span>{</span><br><span class="line"> <span class="variable">$data</span>[<span class="string">'id'</span>] = <span class="number">99</span>;</span><br><span class="line"> <span class="variable">$data</span>[<span class="string">'code'</span>] = <span class="number">123456789</span>;</span><br><span class="line"> <span class="variable">$Test</span> = <span class="title function_ invoke__">M</span>(<span class="string">'test'</span>);</span><br><span class="line"> <span class="variable">$Test</span> -> <span class="title function_ invoke__">where</span>(<span class="string">'id=1'</span>) -> <span class="title function_ invoke__">delete</span>();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>以上代码的方法对应着CURD操作，当路由匹配到方法时，对数据表进行相应的操作。</p><p>当不使用$_GET，而是直接将url内的参数传入到控制器内的方法时，<br>首先，在配置文件中添加</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="string">'URL_PARAMS_BIND'</span> => <span class="literal">true</span>,</span><br></pre></td></tr></table></figure><p>开启参数绑定。<br>以上述cre()函数为例,</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">cre</span>(<span class="params"><span class="variable">$id</span>,<span class="variable">$code</span></span>)</span>{</span><br><span class="line"> <span class="variable">$data</span>[<span class="string">'id'</span>] = <span class="variable">$id</span>;</span><br><span class="line"> <span class="variable">$data</span>[<span class="string">'code'</span>] = <span class="variable">$code</span>;</span><br><span class="line"> <span class="variable">$Test</span> = <span class="title function_ invoke__">M</span>(<span class="string">'test'</span>);</span><br><span class="line"> <span class="variable">$Test</span> -> <span class="title function_ invoke__">where</span>(<span class="string">'id=2'</span>) -> <span class="title function_ invoke__">save</span>(<span class="variable">$data</span>);</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>访问 <a href="http://localhost/index.php/Home/Index/ret/id/01/code/12345">http://localhost/index.php/Home/Index/ret/id/01/code/12345</a> 时，即可传入相应的参数。<br>当然，路由规则也需要加入相应的关键字。</p><p>这里简单说一下内置D方法和M方法的区别。</p><ul><li>当使用THINKPHP模型中的自动验证、关联模型等复杂业务逻辑功能时，使用D方法。</li><li>当仅使用CURD操作时，使用M方法。</li></ul><h3 id="四、模型"><a href="#四、模型" class="headerlink" title="四、模型"></a>四、模型</h3><p>模型类文件创建在Application/Home/Model/下，同样采用首字母大写，驼峰规则，以<br>.class.php结尾命名。注意文件名要与控制器类的名称一致。<br>打开文件添加代码，</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">namespace</span> <span class="title class_">Home</span>\<span class="title class_">Model</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Think</span>\<span class="title">Model</span>;</span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">TestModel</span> <span class="keyword">extends</span> <span class="title">Model</span> </span>{</span><br><span class="line"> <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">get_coupon</span>(<span class="params"></span>)</span>{</span><br><span class="line"> <span class="variable">$arr</span> = <span class="variable language_">$this</span> -> <span class="title function_ invoke__">select</span>();</span><br><span class="line"> <span class="keyword">return</span> <span class="variable">$arr</span>;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>因为在控制器类中使用了get_coupon()方法，所以这里要把它加进去。如果有其他对模型操作的方法都可以写进来，当然写在控制器类也可以，但是为了代码功能明晰，建议分开写。</p><p>数据库连接参数写入配置文件，</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="string">'DB_TYPE'</span> => <span class="string">'mysql'</span>,</span><br><span class="line"><span class="string">'DB_USER'</span> => <span class="string">'root'</span>,</span><br><span class="line"><span class="string">'DB_PWD'</span> => <span class="string">'1234'</span>,</span><br><span class="line"><span class="string">'DB_HOST'</span> => <span class="string">'127.0.0.1'</span>,</span><br><span class="line"><span class="string">'DB_PORT'</span> => <span class="string">'3306'</span>,</span><br><span class="line"><span class="string">'DB_NAME'</span> => <span class="string">'coupon'</span>, <span class="comment">//这是测试用的数据表</span></span><br></pre></td></tr></table></figure><p>开发文档中提到了可以将配置信息生成为数组，但是不在模型类调用这一数组时，不要将配置信息组合为数组。</p><p>测试表coupon内容如下：</p><table><thead><tr><th>id</th><th>code</th></tr></thead><tbody><tr><td>0</td><td>SUFFLzS</td></tr><tr><td>1</td><td>hZxakCl</td></tr><tr><td>2</td><td>xD2gz0N</td></tr><tr><td>3</td><td>6WwLt7U</td></tr><tr><td>4</td><td>xLSxY1n</td></tr><tr><td>5</td><td>bn422Ka</td></tr><tr><td>6</td><td>55BaQ0C</td></tr><tr><td>7</td><td>Ui1Jc8P</td></tr></tbody></table><h3 id="五、模板"><a href="#五、模板" class="headerlink" title="五、模板"></a>五、模板</h3><p>模型类文件创建在Application/Home/View/Index/下,与控制器类内代码相对应，新建main.html文件，添加以下代码：</p><figure class="highlight html"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag"><<span class="name">html</span>></span></span><br><span class="line"><span class="tag"><<span class="name">head</span>></span></span><br><span class="line"><span class="tag"><<span class="name">title</span>></span>测试主页<span class="tag"></<span class="name">title</span>></span></span><br><span class="line"><span class="tag"></<span class="name">head</span>></span></span><br><span class="line"><span class="tag"><<span class="name">body</span>></span></span><br><span class="line"><span class="tag"><<span class="name">foreach</span> <span class="attr">name</span>=<span class="string">'temp'</span> <span class="attr">item</span>=<span class="string">'coupon'</span>></span></span><br><span class="line">{$coupon.id}:{$coupon.code}<span class="tag"><<span class="name">br</span>/></span></span><br><span class="line"><span class="tag"></<span class="name">foreach</span>></span></span><br><span class="line"><span class="tag"></<span class="name">body</span>></span></span><br><span class="line"><span class="tag"></<span class="name">html</span>></span></span><br></pre></td></tr></table></figure><p><code><foreach></code>是内置的标签，可以将数据循环输出。其中temp是控制器类内的数组名称，coupon为自定义名称，id和code为数组内的键名。</p><p>在浏览器地址栏访问 <a href="http://localhost/index.php/Home/Index/cre">http://localhost/index.php/Home/Index/cre</a> 时，<br>页面显示为</p><p>0:SUFFLzS<br>1:hZxakCl<br>2:xD2gz0N<br>3:6WwLt7U<br>4:xLSxY1n<br>5:bn422Ka<br>6:55BaQ0C<br>7:Ui1Jc8P</p><h3 id="六、总结与吐槽"><a href="#六、总结与吐槽" class="headerlink" title="六、总结与吐槽"></a>六、总结与吐槽</h3><p>以下是总结：<br> 至此，THINKEPHP 3.2的快速入门教程已经完成。本文对路由规则分配，控制器操作模型、渲染模板进行了简单的介绍。如果想深入了解框架的其他知识，还请移步官方文档。至于MySQL的相关知识，还是另开一篇文章好了。</p><p>以下是吐槽：<br> “什么AUIMD方法真的是反人类啊，不能顾名思义的方法都不是好方法！”，一位绞尽脑汁想不到新变量名称的程序员说道。</p>]]></content>
<summary type="html">THINKPHP是一个快速简单的轻量级国产PHP开发框架。</summary>
<category term="Hello world" scheme="https://catinsides.github.io/tags/Hello-world/"/>
<category term="PHP" scheme="https://catinsides.github.io/tags/PHP/"/>
</entry>
</feed>