Spring boot Websocket & SseEmitter

现代浏览器已经普遍支持WebSocket和EventSource，可以用它们实现与服务器的实时通信。
WebSocket复杂些，但是双工的；EventSource相对简单且能自动重连，但仅支持服务端推。

WebSocket 配置

Spring boot加入下面的依赖即可使用WebSocket

1
2
3
4
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-websocket</artifactId>
</dependency>

WebSocketConfig.class

注册 Websocket Handler & Interceptor

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
@Configuration
@EnableWebSocket
public class WebSocketConfig implements WebSocketConfigurer {
  @Bean
  public TextWebSocketHandler myWebSocketHandler() {
    return new MyWebSocketHandler();
  }

  @Override
  public void registerWebSocketHandlers(WebSocketHandlerRegistry registry) {
    registry.addHandler(myWebSocketHandler(), "/myweb/socket").addInterceptors(new WebSocketInterceptor()).setAllowedOrigins("*");//https://www.cnblogs.com/exmyth/p/11720371.html  
    //registry.addHandler(myWebSocketHandler(), "/myweb/sockjs").addInterceptors(new WebSocketInterceptor()).withSockJS();  
  }
  
  @Bean
  public TaskScheduler taskScheduler() {//避免找不到TaskScheduler Bean
    ThreadPoolTaskScheduler taskScheduler = new ThreadPoolTaskScheduler();
    taskScheduler.setPoolSize(10);
    taskScheduler.initialize();
    return taskScheduler;
  }
}

WebSocketInterceptor.class

1
2
3
4
5
6
7
8
9
10
11
12
13
public class WebSocketInterceptor extends HttpSessionHandshakeInterceptor {
  @Override
  public boolean beforeHandshake(ServerHttpRequest request, ServerHttpResponse response, WebSocketHandler wsHandler, Map<String, Object> attributes) throws Exception {
    String channel = ((ServletServerHttpRequest)request).getServletRequest().getParameter("ch");
    attributes.put("channel", channel);//传参
    return super.beforeHandshake(request, response, wsHandler, attributes);
  }

  @Override
  public void afterHandshake(ServerHttpRequest request, ServerHttpResponse response, WebSocketHandler wsHandler, Exception ex) {
    super.afterHandshake(request, response, wsHandler, ex);
  }
}

MyWebSocketHandler.class

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
@Slf4j
public class MyWebSocketHandler extends TextWebSocketHandler{
  @Autowired MyWebSocketService myWebSocketService;//注入需要的Service

  @Override
  public void afterConnectionEstablished(WebSocketSession session) throws Exception {
    String channel = (String)session.getAttributes().get("channel");//获取参数
    //记下session和参数用于下一步发消息...
  }

  @Override
  public void afterConnectionClosed(WebSocketSession session, CloseStatus status) throws Exception {
    String channel = (String)session.getAttributes().get("channel");
    //做会话关闭后的处理...
  }

  @Override
  protected void handleTextMessage(WebSocketSession session, TextMessage message) throws Exception {
    log.debug("receive text message: " + message.getPayload());
    //收到消息的处理...
  }

  public void send(WebSocketSession session, String text) {
    try {
      TextMessage message = new TextMessage(text);
      session.sendMessage(message);//发送消息的方法
    } catch (Exception e) {
      e.printStackTrace();
    }
  }
}

SseEmitter

Controller方法返回SseEmitter对象即可为客户端提供EventSource

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
private static Set<SseEmitter> emitters = new HashSet<>();
@RequestMapping("/myweb/eventsource")
@ResponseBody
SseEmitter eventSource(String ch) {
  SseEmitter emitter = new SseEmitter(0L);
  emitters.put(emitter);//记下emitter用于之后发送数据
  emitter.onCompletion(() -> {
    emitters.remove(emitter);//做连接关闭后的处理(ch, emitter)...
  });
  emitter.onTimeout(() -> {
    emitter.complete();
  });
  emitter.onError((e) -> {
    emitter.completeWithError(e);
  });
  return emitter;
}

向所有的emitters发送数据text

1
2
3
4
5
6
7
8
SseEventBuilder builder = SseEmitter.event().data(text);
emitters.forEach(emitter -> {
  try {
    emitter.send(builder);
  } catch (Exception e) {
    errors.add(emitter);
  }
});

客户端连接

前端js对象WebSocket和EventSource分别用于连接这两种服务。
具体用法略。

Nginx需要的额外配置

EventSource

1
2
3
4
5
6
proxy_http_version 1.1;
proxy_set_header Connection '';
proxy_buffering off;
proxy_cache off;
gzip off;
chunked_transfer_encoding off;

WebSocket

1
2
3
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";

已知问题

  1. 火狐下EventSource中断之后不会自动重连。
  2. IE系列浏览器都不支持EventSource。

Java poi 读写excel工具类

依赖

1
2
3
4
5
6
7
8
9
10
<dependency>
	<groupId>org.apache.poi</groupId>
	<artifactId>poi</artifactId>
	<version>3.14</version>
</dependency>
<dependency>
	<groupId>org.apache.poi</groupId>
	<artifactId>poi-ooxml</artifactId>
	<version>3.14</version>
</dependency>

WorkbookUtils.java

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import org.apache.poi.hssf.usermodel.HSSFCellStyle;
import org.apache.poi.hssf.usermodel.HSSFFont;
import org.apache.poi.hssf.usermodel.HSSFWorkbook;
import org.apache.poi.hssf.util.HSSFColor;
import org.apache.poi.ss.usermodel.Cell;
import org.apache.poi.ss.usermodel.CellStyle;
import org.apache.poi.ss.usermodel.DataFormatter;
import org.apache.poi.ss.usermodel.DateUtil;
import org.apache.poi.ss.usermodel.Font;
import org.apache.poi.ss.usermodel.FormulaEvaluator;
import org.apache.poi.ss.usermodel.Row;
import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.ss.usermodel.Workbook;
import org.apache.poi.ss.util.WorkbookUtil;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import org.apache.tomcat.util.http.fileupload.IOUtils;

public class WorkbookUtils extends WorkbookUtil {
	// 读取excel文件
	public static Workbook readExcel(String filePath) {
		Workbook wb = null;
		if (filePath == null) {
			return null;
		}
		String extString = filePath.substring(filePath.lastIndexOf("."));
		InputStream is = null;
		try {
			is = new FileInputStream(filePath);
			if (".xls".equals(extString)) {
				return wb = new HSSFWorkbook(is);
			} else if (".xlsx".equals(extString)) {
				return wb = new XSSFWorkbook(is);
			} else {
				return wb = null;
			}

		} catch (FileNotFoundException e) {
			e.printStackTrace();
		} catch (IOException e) {
			e.printStackTrace();
		} finally {
			IOUtils.closeQuietly(is);
		}
		return wb;
	}

	//读取单元格
	public static Object getCellFormatValue(Cell cell, FormulaEvaluator formulaEvaluator) {
		if (cell == null) {
			return null;
		}
		Object cellValue = null;
		// 判断cell类型
		int cellType = cell.getCellType();
		if (cellType == Cell.CELL_TYPE_FORMULA) {
			cellType = formulaEvaluator.evaluateFormulaCell(cell);
		}
		switch (cellType) {
		case Cell.CELL_TYPE_STRING:
			cellValue = cell.getRichStringCellValue().getString();
			break;
		case Cell.CELL_TYPE_NUMERIC:
			if (DateUtil.isCellDateFormatted(cell)) {// 判断cell是否为日期格式
				cellValue = cell.getDateCellValue();
				break;
			}
			DataFormatter dataFormatter = new DataFormatter();
			cellValue = dataFormatter.formatCellValue(cell, formulaEvaluator);
			break;
		case Cell.CELL_TYPE_BOOLEAN:
			cellValue = cell.getBooleanCellValue();
			break;
		default:
			cellValue = "";
		}
		return cellValue;
	}

	// 设置报表头样式
	public static CellStyle createHeadSytle(Workbook workbook) {
		CellStyle style1 = workbook.createCellStyle();// cell样式
		// 设置单元格背景色，设置单元格背景色以下两句必须同时设置
		style1.setFillPattern(HSSFCellStyle.SOLID_FOREGROUND);// 设置填充样式
		style1.setFillForegroundColor(HSSFColor.GREY_25_PERCENT.index);// 设置填充色
		// 设置单元格上、下、左、右的边框线
		style1.setBorderBottom(HSSFCellStyle.BORDER_THIN);
		style1.setBorderLeft(HSSFCellStyle.BORDER_THIN);
		style1.setBorderRight(HSSFCellStyle.BORDER_THIN);
		style1.setBorderTop(HSSFCellStyle.BORDER_THIN);
		Font font1 = workbook.createFont();// 创建一个字体对象
		font1.setBoldweight((short) 10);// 设置字体的宽度
		font1.setFontHeightInPoints((short) 10);// 设置字体的高度
		font1.setBoldweight(HSSFFont.BOLDWEIGHT_BOLD);// 粗体显示
		style1.setFont(font1);// 设置style1的字体
		// style1.setWrapText(true);// 设置自动换行
		style1.setAlignment(HSSFCellStyle.ALIGN_CENTER);// 设置单元格字体显示居中（左右方向）
		style1.setVerticalAlignment(HSSFCellStyle.VERTICAL_CENTER);// 设置单元格字体显示居中(上下方向)
		return style1;
	}

	// 设置报表体样式
	public static CellStyle createCellStyle(Workbook wb) {
		// 设置style1的样式，此样式运用在第二行
		CellStyle style1 = wb.createCellStyle();// cell样式
		// 设置单元格上、下、左、右的边框线
		style1.setBorderBottom(HSSFCellStyle.BORDER_THIN);
		style1.setBorderLeft(HSSFCellStyle.BORDER_THIN);
		style1.setBorderRight(HSSFCellStyle.BORDER_THIN);
		style1.setBorderTop(HSSFCellStyle.BORDER_THIN);
		// style1.setWrapText(true);// 设置自动换行
		style1.setAlignment(HSSFCellStyle.ALIGN_LEFT);// 设置单元格字体显示居中（左右方向）
		style1.setVerticalAlignment(HSSFCellStyle.VERTICAL_CENTER);// 设置单元格字体显示居中(上下方向)
		return style1;
	}
}

Java commons-exec 执行外部命令

Java创建子进程(Process)执行外部命令底层的方法是new ProcessBuilder().start()或Runtime.getRuntime().exec()。

Apache commons-exec对底层进行封装，提供了更加详细的设置和监控方法。

pom.xml

1
2
3
4
5
<dependency>
	<groupId>org.apache.commons</groupId>
	<artifactId>commons-exec</artifactId>
	<version>1.3</version>
</dependency>

CmdHelper.java

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
import java.io.IOException;
import java.io.OutputStream;
import java.util.concurrent.CompletableFuture;

import org.apache.commons.exec.CommandLine;
import org.apache.commons.exec.DefaultExecutor;
import org.apache.commons.exec.ExecuteWatchdog;
import org.apache.commons.exec.PumpStreamHandler;
import org.apache.commons.exec.ShutdownHookProcessDestroyer;

public class CmdHelper {
	/**
	 * 执行外部命令，等待返回结果
	 * 
	 * @param commandLine: 命令行
	 * @param out:         输出流，为空默认标准输出
	 * @param timeout:     超时，不大于0则不限超时
	 * @return CmdHandler based on DefaultExecuteResultHandler
	 */
	public static CmdHandler run(CommandLine commandLine, OutputStream out, long timeout) {
		PumpStreamHandler pumpStreamHandler = null;
		if (null == out) {
			pumpStreamHandler = new PumpStreamHandler();
		} else {
			pumpStreamHandler = new PumpStreamHandler(out);
		}

		DefaultExecutor executor = new DefaultExecutor();
		CmdHandler cmdHandler = new CmdHandler(executor);
		executor.setStreamHandler(pumpStreamHandler);
		ShutdownHookProcessDestroyer processDestroyer = new ShutdownHookProcessDestroyer();
		executor.setProcessDestroyer(processDestroyer);// 随主进程退出

		if (timeout <= 0) {
			timeout = ExecuteWatchdog.INFINITE_TIMEOUT;
		}
		ExecuteWatchdog watchdog = new ExecuteWatchdog(timeout);
		executor.setWatchdog(watchdog);// 控制超时

		try {
			executor.execute(commandLine, cmdHandler);
			cmdHandler.waitFor();// 等待返回结果
		} catch (IOException | InterruptedException e) {
			e.printStackTrace();
		}
		return cmdHandler;
	}

	/**
	 * 异步执行外部命令
	 * 
	 * @param commandLine: 命令行
	 * @param out:         输出流，为空默认标准输出
	 * @param timeout:     超时，不大于0则不限超时
	 * @return CompletableFuture<CmdHandler>
	 */
	public static CompletableFuture<CmdHandler> exec(CommandLine commandLine, OutputStream out, long timeout) {
		CompletableFuture<CmdHandler> cf = new CompletableFuture<>();
		PumpStreamHandler pumpStreamHandler = null;
		if (null == out) {
			pumpStreamHandler = new PumpStreamHandler();
		} else {
			pumpStreamHandler = new PumpStreamHandler(out);
		}

		DefaultExecutor executor = new DefaultExecutor();
		CmdHandler cmdHandler = new CmdHandler(executor);
		cmdHandler.setCallback(() -> {
			cf.complete(cmdHandler);// 执行完成后回调
		});
		executor.setStreamHandler(pumpStreamHandler);
		ShutdownHookProcessDestroyer processDestroyer = new ShutdownHookProcessDestroyer();
		executor.setProcessDestroyer(processDestroyer);// 随主进程退出

		if (timeout <= 0) {
			timeout = ExecuteWatchdog.INFINITE_TIMEOUT;
		}
		ExecuteWatchdog watchdog = new ExecuteWatchdog(timeout);
		executor.setWatchdog(watchdog);// 控制超时

		try {
			executor.execute(commandLine, cmdHandler);
		} catch (IOException e) {
			e.printStackTrace();
		}
		return cf;
	}

	public static void main(String[] args) throws InterruptedException {
		CommandLine command = CommandLine.parse("ping 127.0.0.1 -t");

		// 测试同步执行
		CmdHandler result = CmdHelper.run(command, null, 3000);
		System.out.println(result.resultString());

		// 测试异步执行
		CmdHelper.exec(command, null, 0).thenAccept(cmdHandler -> {
    		System.out.println(cmdHandler.resultString());
		});
	}
}

CmdHandler.java

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
import org.apache.commons.exec.DefaultExecuteResultHandler;
import org.apache.commons.exec.ExecuteException;
import org.apache.commons.exec.ExecuteWatchdog;
import org.apache.commons.exec.Executor;

public class CmdHandler extends DefaultExecuteResultHandler {
	Executor executor;
	Runnable callback;
	
	public CmdHandler(Executor executor) {
		this.executor = executor;
	}
	
	public void setCallback(Runnable callback) {
		this.callback = callback;
	}
	
	public Executor getExecutor() {
		return this.executor;
	}
	
	public ExecuteWatchdog getWatchdog() {
		if (this.executor == null) return null;
		return this.executor.getWatchdog();
	}
	
	public String resultString() {
		String retMsg = "complete";
		if (this.getException() != null) {
			ExecuteWatchdog watchdog = this.getWatchdog();
			if (watchdog != null && watchdog.killedProcess()) {
				retMsg = "timeout";
			} else {
				retMsg = this.getException().getMessage();
			}
		}
		return this.getExitValue() + ":" + retMsg;
	}

	@Override
	public void onProcessComplete(int exitValue) {
		super.onProcessComplete(exitValue);
		if (callback != null) {
			callback.run();
		}
	}

	@Override
	public void onProcessFailed(ExecuteException e) {
		super.onProcessFailed(e);
		if (callback != null) {
			callback.run();
		}
	}
}

参考

  1. https://www.cnblogs.com/kingcucumber/p/3180146.html
  2. https://www.jianshu.com/p/73aaec23009d

java微信开发常用方法

WeixinService.java

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
public class WeixinSercice {
	final static String URL_SNS_TOKEN = "https://api.weixin.qq.com/sns/oauth2/access_token?appid=%s&secret=%s&code=%s&grant_type=authorization_code";
	final static String URL_ACCESS_TOKEN = "https://api.weixin.qq.com/cgi-bin/token?grant_type=client_credential&appid=%s&secret=%s";
	final static String URL_JSAPI_TICKET = "https://api.weixin.qq.com/cgi-bin/ticket/getticket?access_token=%s&type=jsapi";
										
	final static String URL_ORDER_QUERY = "https://api.mch.weixin.qq.com/pay/orderquery";
	final static String URL_UNIFIED_ORDER = "https://api.mch.weixin.qq.com/pay/unifiedorder";

	public static final String TRADE_TYPE_H5 = "MWEB";
	public static final String TRADE_TYPE_JS = "JSAPI";

	final static String KEY_SIGN = "sign";
	
	@Value("${weixin.appid}")
	private String appId;
	@Value("${weixin.secret}")
	private String appSecret;
	@Value("${weixin.mch_id}")
	private String mchId;
	@Value("${weixin.mch_key}")
	private String mchKey;
	@Value("${weixin.notify_url}")
	private String notifyUrl;
	
	@Autowired
	RestTemplate restTemplate;
    @Autowired
	OrderService orderService;
	@Resource
	private CacheManager cacheManager;
	private static final String CACHE_NAME_WEIXIN = "myapp:weixin";
	private static final String CACHE_KEY_ACCESS_TOKEN = "actoken";
	private static final String CACHE_KEY_JSAPI_TICKET = "jsticket";
	private Cache getCache() {
		return cacheManager.getCache(CACHE_NAME_WEIXIN);
	}

	// 微信网页认证：通过code获取token
	public WxWebToken fetchWebTokenByCode(String code) {
		String url = String.format(URL_SNS_TOKEN, appId, appSecret, code);
		String content = restTemplate.getForObject(url, String.class);
		WxWebToken token = JSON.parse(content, WxWebToken.class);
		return token;
	}

	// 微信JSSDK：获取指定url的config
	public WxJsdkConfig genJsdkConfig(String url) {
		WxJsdkConfig config = new WxJsdkConfig();
		config.setAppId(appId);
		config.setNonceStr(StringUtils.uuid());
		config.setTimestamp(DateTimeUtils.secondsOf(LocalDateTime.now()));
		
		String ticket = getJsapiTicket(url);
		if (StringUtils.isNotBlank(ticket)) {
			String string = "jsapi_ticket=" + ticket +
					"&noncestr=" + config.getNonceStr() +
					"&timestamp=" + config.getTimestamp() +
					"&url=" + url;
			String signature = StringUtils.SHA1(string);
			
			if (StringUtils.isNotBlank(signature)) {
				config.setSignature(signature);
			}
		} else {
			config.setErrcode(1);
			config.setErrmsg("invalid ticket");
		}
		log.debug("genJsdkConfig for {} return {}", url, config.getSignature());
		return config;
	}
	
	//get cached jsapi_ticket
	private String getJsapiTicket(String url) {
		String cacheKey = CACHE_KEY_JSAPI_TICKET+StringUtils.MD5(url);
		WxJsapiTicket ticket = getCache().get(cacheKey, WxJsapiTicket.class);
		if (ticket == null || ticket.getExpired()) {
			ticket = fetchJsapiTicket();
			if (ticket != null && !ticket.getExpired()) {
				getCache().put(cacheKey, ticket);
			}else {
				return null;
			}
		} else {
			log.debug("return cached {} for key {}", CACHE_NAME_WEIXIN, cacheKey);
		}
		return ticket.getTicket();
	}
	
	private WxJsapiTicket fetchJsapiTicket() {
		String accessToken = getAccessToken();
		if (StringUtils.isBlank(accessToken)) {
			return null;
		}
		
		String url = String.format(URL_JSAPI_TICKET, accessToken);
		String content = restTemplate.getForObject(url, String.class);
		WxJsapiTicket ticket = JSON.parse(content, WxJsapiTicket.class);
		if (ticket != null) {
			Long expires = ticket.getExpires_in();
			if (expires != null) {//把过期秒数转化为世纪秒
				expires += DateTimeUtils.secondsOf(LocalDateTime.now());
			}else{
				expires = 0L;
			}
			ticket.setExpires_in(expires);
			log.debug("fetchJsapiTicket return {}-{}", ticket.getErrcode(), ticket.getErrmsg());
		}
		return ticket;
	}
	
	// get cached access_token
	private String getAccessToken() {
		WxAccessToken token = getCache().get(CACHE_KEY_ACCESS_TOKEN, WxAccessToken.class);
		if (token == null || token.getExpired()) {
			token = fetchAccessToken();
			if (token != null && !token.getExpired()) {
				getCache().put(CACHE_KEY_ACCESS_TOKEN, token);
			}
		} else {
			log.debug("return cached {} for key {}", CACHE_NAME_WEIXIN, CACHE_KEY_ACCESS_TOKEN);
		}
		return token.getAccess_token();
	}
	
	private WxAccessToken fetchAccessToken() {
		String url = String.format(URL_ACCESS_TOKEN, appId, appSecret);
		String content = restTemplate.getForObject(url, String.class);
		WxAccessToken token = JSON.parse(content, WxAccessToken.class);
		if (token != null) {
			Long expires = token.getExpires_in();
			if (expires != null) {//把过期秒数转化为世纪秒
				expires += DateTimeUtils.secondsOf(LocalDateTime.now());
			}else{
				expires = 0L;
			}
			token.setExpires_in(expires);
			if (StringUtils.isNotBlank(token.getErrmsg())){
				log.debug("fetchAccessToken return {}-{}", token.getErrcode(), token.getErrmsg());
			}else{
				log.debug("fetchAccessToken return {}", token.getAccess_token());
			}
		} else {
			log.debug("fetchAccessToken return null");
		}
		return token;
	}
	
	
	//处理订单
	public Order processOrder(Order order) {
		if (StringUtils.isNotBlank(order.getId())) {
			Order dbOrder = orderService.findById(order.getId());
			if (dbOrder != null && dbOrder.getStatus()>=Order.STATUS_PAYED) {
				return dbOrder;//已支付
			}
		}
		order = orderService.upsert(order);
		Map<String, String> map = placeOrder(order);
		if (map != null) {
			String returnCode = map.get("return_code");
			order.setReturnCode(returnCode);
			if ("SUCCESS".equals(returnCode)) {
				order.setStatus(Order.STATUS_ORDER);
			}
			order.setReturnMsg(map.get("return_msg"));
			order.setMwebUrl(map.get("mweb_url"));
			order.setPrepayId(map.get("prepay_id"));
			orderService.save(order);
			orderService.sendNotify(order);
			if (TRADE_TYPE_JS.equals(order.getTradeType())) {
				Map<String, String> signs = new TreeMap<>();
				signs.put("appId", appId);
				signs.put("nonceStr", StringUtils.uuid());
				signs.put("package", "prepay_id="+map.get("prepay_id"));
				signs.put("signType", "MD5");
				signs.put("timeStamp", String.valueOf(DateTimeUtils.secondsOf(LocalDateTime.now())));
				signs.put("paySign", genSign(signs));
				order.setSigns(signs);
			}
		}
		return order;
	}

	//下单
	private Map<String, String> placeOrder(Order order) {
	    Map<String, String> map = null;
	    String tradeType = order.getTradeType();
	    if (TRADE_TYPE_H5.equals(tradeType)) {
	    	map = prepareH5Order(order.getTradeNo(), order.getProductId(), order.getProductName(), order.getTotalFee(), order.getIpaddr());
	    }else{
	    	map = prepareJsOrder(order.getTradeNo(), order.getProductId(), order.getProductName(), order.getTotalFee(), order.getIpaddr(), order.getOpenid());
	    }
	    HttpEntity<String> request = genXmlRequest(map);
	    String res = restTemplate.postForObject(URL_UNIFIED_ORDER, request, String.class);
	    log.debug(res);
	    return XmlUtils.parse(res);
	}
	
	// H5支付下单数据
	private Map<String, String> prepareH5Order(String tradeNo, String productId, String productName, Long amount, String ip) {
		Map<String, String> order = newOrderMap();
		order.put("trade_type", TRADE_TYPE_H5);//H5支付的交易类型为MWEB
		order.put("notify_url", notifyUrl);//回调地址, 不能携带参数。
		order.put("scene_info", "{\"h5_info\": {\"type\":\"WAP\",\"wap_url\": \"\",\"wap_name\": \"\"}}");//用于上报支付的场景信息
		order.put("spbill_create_ip", ip);//用户端IP,支持ipv4、ipv6格式
		order.put("out_trade_no", tradeNo);//自定义交易单号
		order.put("product_id", productId);//自定义商品
		order.put("body", productName);//网页的主页title名-商品概述
		order.put("fee_type", "CNY");//境内只支持CNY，默认可不传
		order.put("total_fee", String.valueOf(amount));//订单总金额，单位为分
		//签名
		order.put(KEY_SIGN, genSign(order));
		return order;
	}
	
	// JSAPI支付下单数据
	private Map<String, String> prepareJsOrder(String tradeNo, String productId, String productName, Long amount, String ip, String openid) {
		Map<String, String> order = newOrderMap();
		order.put("trade_type", TRADE_TYPE_JS);//交易类型为JSAPI
		order.put("notify_url", notifyUrl);//回调地址, 不能携带参数。
		//order.put("scene_info", "{\"h5_info\": {\"type\":\"WAP\",\"wap_url\": \"\",\"wap_name\": \"\"}}");//用于上报支付的场景信息
		order.put("openid", openid);
		order.put("spbill_create_ip", ip);//用户端IP,支持ipv4、ipv6格式
		order.put("out_trade_no", tradeNo);//自定义交易单号
		order.put("product_id", productId);//自定义商品
		order.put("body", productName);//网页的主页title名-商品概述
		order.put("fee_type", "CNY");//境内只支持CNY，默认可不传
		order.put("total_fee", String.valueOf(amount));//订单总金额，单位为分
		//签名
		order.put(KEY_SIGN, genSign(order));
		return order;
	}

	// 下单数据准备：公用部分
	private Map<String, String> newOrderMap() {
		Map<String, String> order = new TreeMap<>();
		order.put("appid", appId);
		order.put("mch_id", mchId);
		order.put("nonce_str", StringUtils.uuid());
		return order;
	}
	
	// 构造xml request
	private HttpEntity<String> genXmlRequest(Object data) {
		HttpHeaders headers = new HttpHeaders();
	    headers.setContentType(MediaType.APPLICATION_XML);
	    String body = XmlUtils.stringify(data, "xml");
	    return new HttpEntity<String>(body, headers);
	}
	
	// 生成签名
	private String genSign(Map<String, String> paramMap) {
		StringBuilder sb = new StringBuilder();
		paramMap.entrySet().stream().sorted(Map.Entry.comparingByKey()).forEach((a) -> {
			if (StringUtils.isBlank(a.getKey()) || KEY_SIGN.equals(a.getKey())) {
				return;
			}
			if (StringUtils.isBlank(a.getValue())) {
				return;
			}
			sb.append(a.getKey()); sb.append("="); sb.append(a.getValue()); sb.append("&");
		});
		sb.append("key="); sb.append(mchKey);
		String signStr = sb.toString();
		log.debug(signStr);
		return StringUtils.MD5(signStr).toUpperCase();
	}
	
	//微信支付结果回调处理
	@Synchronized // TODO: 避免重入仅这样不够，还需要锁定订单记录
	public String processCallback(String data) {
		Map<String, String> map = XmlUtils.parse(data);
		String sign = genSign(map);
		if (!sign.equals(map.get(KEY_SIGN))) {
			return returnCodeMsg("FAIL", "SIGNERROR");
		};
		String tradeNo = map.get("out_trade_no");
		Order order = orderService.findByTradeNo(tradeNo);
		if (order == null) {
			return returnCodeMsg("FAIL", "NOTFOUND");
		}
		if (order.getStatus() >= Order.STATUS_PAYED) {
			return returnCodeMsg("SUCCESS", "OK!");
		}
		String tradeType = map.get("trade_type");
		long totalFee = NumberUtils.parse(map.get("total_fee"), Long.class, 0L);
		if (totalFee != order.getTotalFee() || !StringUtils.equals(tradeType, order.getTradeType())) {
			return returnCodeMsg("FAIL", "TRADEINFOERROR");
		}
		
		order.setTransactionId(map.get("transaction_id"));
		order.setReturnCode(map.get("return_code"));
		order.setResultCode(map.get("result_code"));
		order.setBankType(map.get("bank_type"));
		order.setTimeEnd(map.get("time_end"));
		order.setStatus(Order.STATUS_PAYED);
		orderService.save(order);
		
		return returnCodeMsg("SUCCESS", "OK");
	}
	
	private String returnCodeMsg(String code, String msg) {
		return String.format("<xml><return_code><![CDATA[%s]]></return_code><return_msg><![CDATA[%s]]></return_msg></xml>", code, msg);
	}

}

## WxResponse.java
```java
@Data
public class WxResponse implements Serializable {
	private Integer errcode;
	private String errmsg;
}

WxResexpire.java

1
2
3
4
5
6
7
@Data
public class WxResexpire extends WxResponse implements Serializable {
	Long expires_in;
	public Boolean getExpired() {
		return expires_in == null || expires_in <= DateTimeUtils.secondsOf(LocalDateTime.now());
	}
}

WxAccessToken.java

1
2
3
4
@Data
public class WxAccessToken extends WxResexpire implements Serializable {
	String access_token;
}

WxWebToken.java

1
2
3
4
5
6
@Data
public class WxWebToken extends WxAccessToken implements Serializable {
	String refresh_token;
	String openid;
	String scope;
}

WxJsapiTicket.java

1
2
3
4
@Data
public class WxJsapiTicket extends WxResexpire implements Serializable {
	String ticket;
}

WxJsdkConfig.java

1
2
3
4
5
6
7
8
@Data
@JsonInclude(value = Include.NON_NULL)
public class WxJsdkConfig extends WxResponse implements Serializable {
	String appId;
	Long timestamp;
	String nonceStr;
	String signature;
}

Spring boot thymeleaf使用外置template和static路径

Spring boot开发的web项目打jar包部署时，如果希望template模板及static文件(js/css/img等)能单独更新，可以用下面的方法把它们从jar包分离出来。

工程目录结构调整

把static和template移到resources之外，比如和java目录平级。

1
2
3
4
5
6
7
8
├─src
│  ├─main
│  │  ├─java
│  │  ├─resources
│  │  │  └─application.properties
│  │  ├─static
│  │  └─templates
│  └─test

在 application.properties 分别指定static和template的位置

1
2
3
4
spring.mvc.static-path-pattern=/static/**
spring.resources.static-locations=file:/pathto/yourproject/src/main/static/
spring.thymeleaf.prefix=file:/pathto/yourproject/src/main/templates/
#spring.thymeleaf.cache=false

html模板里引用static文件的方式

1
2
3
<link rel="stylesheet" th:href="@{/static/subpath/afile.css}" />
<script th:src="@{/static/subpath/afile.js}"></script>
<img th:src="@{/static/subpath/afile.png}"/>

注：link favicon不知为何不能加static:(

把static和template合并，方便一起编辑

实践中发现，html模板和对应的静态文件(js,css,images等)分开在两处不方便编辑。尝试把他们合并到同一个文件夹下。首先只要修改下面两个配置都指向一个地方(比如webroot)：

1
2
spring.resources.static-locations=file:/pathto/yourproject/src/main/webroot/
spring.thymeleaf.prefix=file:/pathto/yourproject/src/main/webroot/

只是这样会导致html模板也可以被用户直接访问，不太安全。需要调整一下security配置：

  • 配置方法configure(WebSecurity webSecurity) ：
    1
    2
    3
    4
    5
    6
    // 忽略静态资源改为只忽略特定类型的静态资源，目的是不忽略*.html模板
    //webSecurity.ignoring().antMatchers("/static/**");
    webSecurity.ignoring().antMatchers("/static/**/*.js");
    webSecurity.ignoring().antMatchers("/static/**/*.css");
    webSecurity.ignoring().antMatchers("/static/**/*.jpg");
    webSecurity.ignoring().antMatchers("/static/**/*.png");
  • 配置方法configure(HttpSecurity httpSecurity)：
    1
    2
    //禁止直接访问html模板
    httpSecurity.authorizeRequests().antMatchers("/static/**/*.html").denyAll()

Spring boot 处理 error

参考： https://www.cnblogs.com/hyl8218/p/10754894.html
Spring boot 处理 error 的基本流程：
Controller -> 发生错误 -> BasicErrorController -> 根据 @RequestMapping(produces) 判断调用 errorHtml 或者 error 方法，然后：
errorHtml -> getErrorAttributes -> ErrorViewResolver -> 错误显示页面
error -> getErrorAttributes -> @ResponseBody (直接返回JSON)

附：为静态文件加md5（避免浏览器缓存旧文件）

1
2
spring.resources.chain.strategy.content.enabled=true
spring.resources.chain.strategy.content.paths=/**

附一：content-path 相关处理

配置项

1
server.servlet.context-path=/myweb

后端获取basePath

1
2
3
4
5
6
7
public static String getWebBasePath() {
  RequestAttributes requestAttributes = RequestContextHolder.getRequestAttributes();
  if (requestAttributes == null) return null;//防止意外
  HttpServletRequest request = ((ServletRequestAttributes) requestAttributes).getResponse();
  if (request == null) return null;//防止意外
  return String.format("%s://%s:%s%s/", request.getScheme(), request.getServerName(), request.getServerPort(), request.getContextPath());
}

在js里获取contentPath

1
2
3
<script th:inline="javascript">
  var contextPath = /*[[${#request.contextPath}]]*/'';
</script>

注：在html模板里使用th:href或th:src时带”@”符号会自动处理contentPath

附二：为thymeleaf模板设置全局静态变量

以配置第三方库路径为例

配置项

1
basepath.lib=https://cdnjs.cloudflare.com/ajax/

配置 ThymeleafViewResolver

1
2
3
4
5
6
7
8
9
10
11
@Resource
private Environment env; //环境变量

@Resource
private void configureThymeleafStaticVars(ThymeleafViewResolver viewResolver) {
  if(viewResolver != null) {
    Map<String, Object> vars = new HashMap<>();
    vars.put("libBasePath", env.getProperty("basepath.lib"));//静态库配置
    viewResolver.setStaticVariables(vars);
  }
}

使用libBasePath引入第三方库(比如vue.js)

1
<script th:src="${libBasePath}+'libs/vue/2.6.10/vue.js'"></script>

附三：html模板共用head

common.html 里定义公用head

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
<!DOCTYPE html>
<html lang="zh" xmlns="http://www.w3.org/1999/xhtml" xmlns:th="http://www.thymeleaf.org">
<head th:fragment="head">
    <meta charset="utf-8"/>
    <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, shrink-to-fit=no"/>
    <link rel="icon" type="image/x-icon" th:href="@{/favicon.ico}">
    <link rel="stylesheet" th:href="${libBasePath}+'libs/twitter-bootstrap/4.3.1/css/bootstrap.min.css'"/>
    <script th:src="${libBasePath}+'libs/vue/2.6.10/vue.js'"></script>
    <link th:href="@{/static/base.css}" rel="stylesheet"/>
    <script th:src="@{/static/base.js}"></script>
    <script th:inline="javascript">var contentPath = /*[[${#request.contextPath}]]*/'';</script>
</head>
<body>
</body>
<html>

index.html 里引用common::head

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<!DOCTYPE html>
<html lang="zh" xmlns="http://www.w3.org/1999/xhtml" xmlns:th="http://www.thymeleaf.org">
<head>
    <title>我的页面</title>
    <th:block th:include="common::head"></th:block>
    <link rel="stylesheet" th:href="@{/static/index.css}" />
</head> 
<body>
<script th:inline="javascript">
    var varForJs = /*[[${varForJs}]]*/{};
</script>
<script th:src="@{/static/index.js}"></script>
<body>
</html>

附四：使用thymeleaf简单制作vue组件

用tab-bar做例子

tabar.html

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
<!DOCTYPE html>
<html lang="zh" xmlns="http://www.w3.org/1999/xhtml" xmlns:th="http://www.thymeleaf.org">
<body>
<th:block th:fragment="tabar">
<script type="text/x-template" id="tabar-tpl"> 
  <div class="nav nav-tabs">
    <a class="nav-item nav-link" href="#"
        v-for="(tabName,tabKey) in tabs" v-show="!!tabName"
        :class="{active: active == tabKey}"
        @click="onClick(tabKey, tabName)">{{tabName}}</a>
    <slot></slot>
  </div>
</script>
<script th:inline="javascript">
Vue.component('tabar', {
  template: '#tabar-tpl',
  props:{
    tabs: [Object, Array],
    active: [String, Number],
  },
  data() {
    return {}
  },
  methods: {
    onClick(tabKey, tabName){
      this.$emit('click-tab', {key: tabKey, name: tabName})
    },
  }
});
</script>
</th:block>
</body>
</html>

引入和使用

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<head>
  <th:block th:include="tabar::tabar"></th:block>
</head>
<body>
  <div id="root">
    <tabar :tabs="tabs" :active="activeTab" @click-tab="activeTab=$event.key"></tabar>
  </div>
</body>
<script>
new Vue({
  el: '#root',
  data: {
    tabs: {a:'Tab A', b:'Tab B'},
    activeTab: 'a',
  },
  methods: {
  },
});
</script>

Java转码服务

记录用java实现转码服务的核心代码。功能包括：把视频转为mp4、把文档转为pdf、生成视频和文档的缩略图等。

ConvertService.java

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
import java.time.Instant;
import java.time.LocalDateTime;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentLinkedQueue;
import javax.annotation.PostConstruct;
import javax.annotation.PreDestroy;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestTemplate;
import lombok.Synchronized;

@Service
public class ConvertService {
	@Autowired
	ConvertConfig convertConfig;
	@Autowired
	ConvertTaskDao convertTaskDao;
	@Autowired
	ConvertHelper convertHelper;
	@Autowired
	RestTemplate restTemplate;
	
	private static Thread convertThread = null;
	private static Thread notifyThread = null;
	private static ConcurrentLinkedQueue<ConvertTask> queueTasks = new ConcurrentLinkedQueue<>();
	private static ConcurrentHashMap<String, ConvertTask> convertingTasks = new ConcurrentHashMap<>();
	private static ConcurrentLinkedQueue<ConvertTask> notifyTasks = new ConcurrentLinkedQueue<>();
	
	@Synchronized
	public ConvertTask addTask(ConvertTask task) {
		ConvertTask save = convertTaskDao.save(task);
		queueTasks.add(save);
		startConvert();
		return task;
	}
	
	@Synchronized
	private int doConvert() {
		if (convertingTasks.size() >= 4) {
			return 1;
		}
		ConvertTask task = queueTasks.poll();
		if (task == null) {
			return 10;
		}
		try {
			if (convertingTasks.containsKey(task.getSrc())) {//重复
				task.setStatus(ConvertTask.STATUS_IGNORE);
				convertTaskDao.save(task);
				return 0;
			}
			convertingTasks.put(task.getSrc(), task);
			task.setStatus(ConvertTask.STATUS_START);
			task.setStartAt(Instant.now());
			convertTaskDao.save(task);
			
			CompletableFuture<ConvertTask> process = convertHelper.process(task);
			process.thenAccept((retTask) -> {
				retTask.setStatus(ConvertTask.STATUS_FINISH);
				retTask.setEndAt(Instant.now());
				convertTaskDao.save(retTask);
				convertingTasks.remove(task.getSrc());
				notifyTasks.add(retTask);
				startNotify();
				System.out.println(retTask);
			});
		} catch(Exception e) {
			e.printStackTrace();
		}
		return 0;
	}
	
	@Synchronized
	private int doNotify() {
		ConvertTask task = notifyTasks.poll();
		if (task == null) {
			return 10;
		}
		int sleep = 0;
		try {
			ResponseEntity<String> response = restTemplate.postForEntity(convertConfig.getNotify(), task, String.class);
			String success = response.getBody();
			System.out.println(success);
			if (!StringUtils.isNullOrEmpty(success) && success.contains("success")) {
				task.setStatus(ConvertTask.STATUS_NOTIFIED);
				task.setNotify(LocalDateTime.now()+","+success);
			} else {
				task.setNotify(LocalDateTime.now()+",empty");
				notifyTasks.add(task);
				sleep = 1;
			}
		} catch(Exception e) {
			e.printStackTrace();
			task.setNotify(LocalDateTime.now()+","+e.getMessage());
			notifyTasks.add(task);
			sleep = 2;
		}
		convertTaskDao.save(task);
		return sleep;
	}
	
	public void startConvert() {
		if (convertThread != null) {
	        convertThread.interrupt();
			return;
		}
		convertThread = new Thread(() -> {
			while (convertThread == Thread.currentThread()) {
				System.out.println("Convert "+convertThread + " running at " + LocalDateTime.now());
				int sleep = doConvert();
				try {
					Thread.sleep(sleep*1000);
				} catch (InterruptedException e) {
					System.out.println("Convert "+Thread.currentThread() +" "+ e.getMessage());
				}
			}
			System.out.println("Convert "+Thread.currentThread()+" stoped at " + LocalDateTime.now());
		});
		convertThread.start();
		System.out.println("Convert "+convertThread+" started at " + LocalDateTime.now());
		
	}
	
	public void startNotify() {
		if (notifyThread != null) {
			notifyThread.interrupt();
			return;
		}
		notifyThread = new Thread(() -> {
			while (notifyThread == Thread.currentThread()) {
				System.out.println("Notify "+notifyThread + " running at " + LocalDateTime.now());
				int sleep = doNotify();
				try {
					Thread.sleep(sleep*1000);
				} catch (InterruptedException e) {
					System.out.println("Notify "+Thread.currentThread() +" "+ e.getMessage());
				}
			}
			System.out.println("Notify "+Thread.currentThread()+" stoped at " + LocalDateTime.now());
		});
		notifyThread.start();
		System.out.println("Notify "+notifyThread+" started at " + LocalDateTime.now());
	}
	
	@PostConstruct
	private void init() {
		List<ConvertTask> tasks = convertTaskDao.findByStatus(ConvertTask.STATUS_START);
		tasks.forEach(task -> {
			queueTasks.add(task);
		});
		tasks = convertTaskDao.findByStatus(ConvertTask.STATUS_INITIAL);
		tasks.forEach(task -> {
			queueTasks.add(task);
		});
		tasks = convertTaskDao.findByStatus(ConvertTask.STATUS_FINISH);
		tasks.forEach(task -> {
			notifyTasks.add(task);
		});
		startConvert();
		startNotify();
		System.out.println(convertConfig);
	}
	
	@PreDestroy
	public void stop() {
		if (convertThread != null) {
	        convertThread.interrupt();
	        convertThread = null;
		}
		if (notifyThread != null) {
			notifyThread.interrupt();
			notifyThread = null;
		}
	}
}

ConvertTaskDao.java

1
2
3
4
5
6
7
import java.util.List;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.repository.query.Param;

public interface ConvertTaskDao  extends JpaRepository<ConvertTask, Integer>{
	List<ConvertTask> findByStatus(@Param("status") Integer status);
}

ConvertTask实体类

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
public class ConvertTask implements Serializable{
	private static final long serialVersionUID = 1418649194096365375L;
    //状态常量
	public static final Integer STATUS_IGNORE = -1;
	public static final Integer STATUS_INITIAL = 0;
	public static final Integer STATUS_START = 1;
	public static final Integer STATUS_FINISH = 2;
	public static final Integer STATUS_NOTIFIED = 3;
	
	@Id
	Integer id;
	String name;
	Instant createdAt;
	Instant startAt;
	Instant endAt;
	Integer status = STATUS_INITIAL;
	String src;//源文件
	String out;//输出文件
	String img;//缩略图
	String ret;//转换结果
	String notify;//通知结果
}

ConvertHelper.java

基于CmdHelper.java实现，见：https://www.zybuluo.com/TedZhou/note/1716823

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.util.concurrent.CompletableFuture;

import javax.imageio.ImageIO;

import org.apache.commons.exec.CommandLine;
import org.apache.pdfbox.io.IOUtils;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.rendering.PDFRenderer;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

@Component
public class ConvertHelper {
	static final String EXT_PDF = ".pdf";
	static final String EXT_MP4 = ".mp4";
	static final String EXT_AVI = ".avi";
	static final String EXT_DOC_STR = ".doc,.docx,.ppt,.pptx,.xls,.xlsx,";// 文档后缀
	static final String EXT_VIDEO_FFMPEG_STR = ".avi,.mpg,.wmv,.3gp,.mov,.asf,.asx,.flv,.f4v,.vob,.mkv,.ts,";// ffmpeg能解码的文件格式
	static final String EXT_VIDEO_MENCODER_STR = ".wmv9,.rm,.rmvb,.mpeg,";// 需要先mencoder解码的文件格式。.mpeg无法直接截取图片，所以需要中转

	@Autowired
	ConvertConfig convertConfig;
	
	public static String getExt(String filename) {
		if (StringUtils.isNullOrEmpty(filename)) return "";
		int pos = filename.lastIndexOf(".");
		if (pos < 0) return "";
		return filename.substring(pos).toLowerCase();
	}
	
	public static boolean isDoc(String ext) {
		return EXT_DOC_STR.contains(ext + ',');
	}

	public static boolean isVideo(String ext) {
		return EXT_VIDEO_FFMPEG_STR.contains(ext + ',') || needMEncoder(ext);
	}

	public static boolean needMEncoder(String ext) {
		return EXT_VIDEO_MENCODER_STR.contains(ext + ',');
	}

	public CompletableFuture<ConvertTask> process(ConvertTask task) {
		CompletableFuture<ConvertTask> cf = new CompletableFuture<>();
		new Thread(() -> {
			String srcFile = task.getSrc();
			String dotExt = getExt(srcFile);
			String outFile = null;
			CmdHandler cmdHandler = null;
			FileOutputStream logStream = null;
			try {
				File logFile = new File(convertConfig.getLog() + srcFile + ".log");
				logStream = new FileOutputStream(logFile);
				if (isDoc(dotExt)) {
					outFile = srcFile + EXT_PDF;
					cmdHandler = doc2pdf(convertConfig.getSrc() + srcFile, convertConfig.getOut() + outFile, logStream);
					if (cmdHandler.getException() == null) {
						if (pdf2png(convertConfig.getOut() + outFile, convertConfig.getOut() + task.getSrc() + ".png")) {
							task.setImg(task.getSrc() + ".png");
						}
					}
				} else if (isVideo(dotExt)) {
					if (needMEncoder(dotExt)) {
						outFile = srcFile + EXT_AVI;
						cmdHandler = video2avi(convertConfig.getSrc() + srcFile, convertConfig.getOut() + outFile, logStream);
						task.setRet(cmdHandler.resultString());
						if (cmdHandler.getException() != null) {
							cf.complete(task);
							return;
						}
						srcFile = outFile;
					}
					cmdHandler = video2jpg(convertConfig.getSrc() + srcFile, convertConfig.getOut() + task.getSrc() + ".jpg", logStream);
					if (cmdHandler.getException() == null) {
						task.setImg(task.getSrc() + ".jpg");
					}
					outFile = task.getSrc() + EXT_MP4;
					cmdHandler = video2mp4(convertConfig.getSrc() + srcFile, convertConfig.getOut() + outFile, logStream);
				} else {
					task.setRet("-1:unsupport");
					cf.complete(task);
					return;
				}
			} catch (FileNotFoundException e) {
				e.printStackTrace();
			} finally {
				IOUtils.closeQuietly(logStream);
			}

			task.setOut(outFile);
			task.setRet(cmdHandler.resultString());
			if (cmdHandler.getException() != null) {
				task.setOut(null);
			}
			cf.complete(task);
		}).start();
		return cf;
	}

	CmdHandler doc2pdf(String srcFile, String outFile, OutputStream cmdOut) {
		CommandLine cmd = new CommandLine(convertConfig.getUnoconv());
		cmd.addArgument("-f");
		cmd.addArgument("pdf");
		cmd.addArgument("-o");
		cmd.addArgument(outFile);
		cmd.addArgument(srcFile);
		return CmdHelper.run(cmd, cmdOut, 0);
	}

	boolean pdf2png(String srcFile, String outFile) {
		try {
			File file = new File(srcFile);
			PDDocument doc;
			doc = PDDocument.load(file);
			PDFRenderer renderer = new PDFRenderer(doc);
			BufferedImage image = renderer.renderImageWithDPI(1, 144); // Windows native DPI
			BufferedImage tag = new BufferedImage(64, 64, BufferedImage.TYPE_INT_RGB);
			tag.getGraphics().drawImage(image, 0, 0, 64, 64, null);
			ImageIO.write(tag, "PNG", new File(outFile));
			return true;
		} catch (IOException e) {
			e.printStackTrace();
			return false;
		}
	}

	CmdHandler video2mp4(String srcFile, String outFile, OutputStream cmdOut) {
		CommandLine cmd = new CommandLine(convertConfig.getFfmpeg());
		cmd.addArgument("-i");
		cmd.addArgument(srcFile);
		cmd.addArgument("-ar");
		cmd.addArgument("22050");
		cmd.addArgument("-vcodec");
		cmd.addArgument("libx264");
		cmd.addArgument("-q:v");
		cmd.addArgument("6");
		cmd.addArgument("-r");
		cmd.addArgument("25");
		cmd.addArgument("-flags");
		cmd.addArgument("+loop");
		cmd.addArgument("-crf");
		cmd.addArgument("24");
		cmd.addArgument("-bt");
		cmd.addArgument("256k");
		cmd.addArgument("-af");
		cmd.addArgument("volume=2");
		cmd.addArgument("-y");
		cmd.addArgument(outFile);
		return CmdHelper.run(cmd, cmdOut, 0);
	}

	CmdHandler video2jpg(String srcFile, String outFile, OutputStream cmdOut) {
		CommandLine cmd = new CommandLine(convertConfig.getFfmpeg());
		cmd.addArgument("-i");
		cmd.addArgument(srcFile);
		cmd.addArgument("-f");
		cmd.addArgument("image2");
		cmd.addArgument("-ss");
		cmd.addArgument("15");
		cmd.addArgument("-t");
		cmd.addArgument("0.001");
		cmd.addArgument("-s");
		cmd.addArgument("64x64");
		cmd.addArgument("-y");
		cmd.addArgument(outFile);
		return CmdHelper.run(cmd, cmdOut, 0);
	}

	CmdHandler video2avi(String srcFile, String outFile, OutputStream cmdOut) {
		CommandLine cmd = new CommandLine(convertConfig.getMencoder());
		cmd.addArgument(srcFile);
		cmd.addArgument("-oac");
		cmd.addArgument("mp3lame");
		cmd.addArgument("-lameopts");
		cmd.addArgument("preset=64");
		cmd.addArgument("-ovc");
		cmd.addArgument("xvid");
		cmd.addArgument("-xvidencopts");
		cmd.addArgument("bitrate=600");
		cmd.addArgument("-of");
		cmd.addArgument("avi");
		cmd.addArgument("-o");
		cmd.addArgument(outFile);
		return CmdHelper.run(cmd, cmdOut, 0);
	}
}

配置项

1
2
3
4
5
6
7
convert.src=/mnt/file/upload/
convert.out=/mnt/file/download/
convert.log=/mnt/file/download/
convert.ffmpeg=/opt/ffmpeg/ffmpeg
convert.unoconv=/opt/libreoffice6.4/unoconv.py
convert.mencoder=/usr/bin/mencoder
convert.notify=http://127.0.0.1:8080/api/convert/notify
1
2
3
4
5
6
7
8
9
10
11
12
@Data
@Configuration
@ConfigurationProperties(prefix = "convert")
public class ConvertConfig {
	String src;
	String out;
	String log;
	String ffmpeg;
	String unoconv;
	String mencoder;
	String notify;
}

依赖

pom.xml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<dependency>
	<groupId>org.apache.commons</groupId>
	<artifactId>commons-exec</artifactId>
	<version>1.3</version>
</dependency>
<dependency>
	<groupId>org.apache.pdfbox</groupId>
	<artifactId>pdfbox</artifactId>
	<version>2.0.9</version>
</dependency>
<dependency>
	<groupId>org.projectlombok</groupId>
	<artifactId>lombok</artifactId>
</dependency>

ffmpeg && mencoder

1
2
3
4
5
wget https://download1.rpmfusion.org/free/el/rpmfusion-free-release-7.noarch.rpm
yum install epel-release
rpm -Uvh rpmfusion-free-release-7.noarch.rpm
wget https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-amd64-static.tar.xz
yum install mencoder

libreoffice && unoconv

1
2
https://www.libreoffice.org/donate/dl/rpm-x86_64/6.4.4/zh-CN/LibreOffice_6.4.4_Linux_x86-64_rpm.tar.gz
https://codeload.github.com/unoconv/unoconv/zip/0.8.2

简单实现Web页面元素拖放功能(Web Drag & Drop)

之前，想在浏览器中实现拖放似乎很困难，现在有了HTML5就简单了。下面实现一个例子：简单的拖拽排序。

拖放测试页面(DragDrop.html)

页面使用bootstrap展示几张图片(缩略图)
注：省略号部分表示你可以自己在div.row下复制更多个div.thumbnail

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Drag & Drop sample</title>
    <link rel="stylesheet" href="http://cdn.staticfile.org/twitter-bootstrap/3.1.1/css/bootstrap.min.css">
  </head>
  <body>
    <div class="container">
      <div class="row">
        <div class="thumbnail col-sm-6 col-md-4">
            <img class="img-responsive" src="img1.png">
            <div class="caption text-center"><h5>Image1</h5></div>
        </div>
        <div class="thumbnail col-sm-6 col-md-4">
            <img class="img-responsive" src="img2.png">
            <div class="caption text-center"><h5>Image2</h5></div>
        </div>
        <!-- ...... -->
      </div>
    </div>
    <script src="http://cdn.staticfile.org/jquery/1.10.2/jquery.min.js"></script>
    <script src="DragDrop.js"></script>
  </body>
</html>

拖放测试脚本(DragDrop.js)

注:脚本依赖的jquery已经在页面里引入。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
$(function(){
  //拖拽的对象是这些缩略图
  var dragable = '.thumbnail';
  var $dragable = $(dragable);
  //设置可拖拽属性
  $dragable.attr('draggable', true);
  //响应开始拖拽事件
  $dragable.on('dragstart', setDragData);
  //拖放的目标也是这些缩略图
  var $dropable = $dragable;
  //让目标允许拖放
  $dropable.on('dragover', allowDrop);
  //响应拖放事件
  $dropable.on('drop', dropDragData);

  //开始拖拽事件处理
  function setDragData(e){
    var $target = getDragable(e.target);
    //传递被拖拽对象的位置索引
    e.originalEvent.dataTransfer.setData("text",$target.index());
  }
  
  function allowDrop(e){
    e.preventDefault();//这样就能允许拖放
  }
  
  //拖放事件处理：把被拖拽对象放到目标对象所在的位置，即实现拖拽排序。
  function dropDragData(e){
    e.preventDefault();//阻止默认的放置行为
    var $target = getDragable(e.target);
    var dragIdx = e.originalEvent.dataTransfer.getData("text");
    if (dragIdx !== $target.index()){
      var $draged = $(dragable).eq(dragIdx);
      if ($draged.index() > $target.index()){
        $target.before($draged);//向前拖：放在目标对象的前面
      }else{
        $target.after($draged);//向后拖：放在目标对象的后面
      }
    }
  }
  //获取可拖拽对象
  function getDragable(dom){
    var $dragable = $(dom);
    if (!$dragable.is(dragable)){
      //拖拽子元素时定位到父元素
      $dragable = $dragable.closest(dragable);
    }
    return $dragable;
  }
});

总结

上例的拖放效果在最新的Chrome、IE、QQ浏览器下测试了都很正常，但在FireFox下拖放图片会激活一个新的页签（有人说使用e.preventDefault()可以解决，但我试了没有效果）。

让table固定表头并冻结列的一种简单实现方法

标题已经说明需求，就不说废话了，直接上code;)

.html

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
<!--把table放到div容器里。-->
<div class="table-container">
  <table class="table-fixed">
    <thead>
      <tr>
        <th class="col-fixed">fixed0</th>
        <th>column0</th>
        ...省略其它th...
      </tr>
    </thead>
    <tbody>
      <tr>
        <td class="col-fixed">fixed1</td>
        <td>column1</td>
        ....省略其它td...
      </tr>
      ....省略其它tr...
    </tbody>
  </table>
<div>

.css

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
<style>
/*限制容器宽高并允许滚动*/
.table-container{
  width:100%;
  max-height:200px;
  overflow:auto;
}
/*表头cell设为相对定位*/
.table-fixed>thead>tr>th{
  position:relative;
  z-index:100;
  background-color:#eee;
}
/*冻结列设为相对定位*/
.table-fixed>tbody>tr>td.col-fixed{
  position:relative;
  background-color:#eee;
}
/*设置z-index避免被覆盖*/
.table-fixed>thead>tr>th.col-fixed{
  z-index:110;
}
</style>

.js

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
<script>
//需先引入jquery
$(function(){
  var $table = $('.table-fixed');
  var $topFixes = $table.find('thead>tr>th');
  var $colFixes = $table.find('.col-fixed');
  //容器滚动时，为实现fixed冻结效果：
  //1、把表头cell的top设为scrollTop
  //2、冻结列的left设为scrollLeft
  $table.closest('.table-container').scroll(function(){
    $topFixes.css('top',this.scrollTop);
    $colFixes.css('left',this.scrollLeft);
  });
});
</script>

##总结
相较于表头和内容分离的做法，这种实现方式不会出现对不齐的问题，而且比较简单（:不想写js的请忽略）

##已知问题

  1. 桌面Chrome下的效果最好，其他浏览器fixed内容有点闪烁。
  2. 移动端浏览器下的延迟比较严重，只能改为用多个table实现。
  3. 如果冻结列超出容器(.table-container)的范围，会导致水平滚动条滚不到头（无限滚动）。

Mongodb MapReduce 介绍、示例及特别说明

介绍

Map-Reduce是一种计算模型，简单的说就是将大批量的工作（数据）分解（MAP）执行，然后再将结果合并成最终结果（REDUCE）。
MongoDB提供的Map-Reduce非常灵活，可以高效的进行大规模数据的统计分析。

语法

格式一

1
2
3
4
5
6
7
8
9
10
11
12
13
db.collection.mapReduce(
   function() {emit(key,value)},//map 函数
   function(key,values) {return reduced}, //reduce 函数
   <collection>//out collection
)
//第三个参数也可以传入更多选项：
{
    finalize: function(key, reduced){return finalized}
    out: <collection>|{inline:true},
    query: <document>,
    sort: <document>,
    limit: <number>,
}

格式二

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
db.runCommand({
     mapReduce: <collection>,
     map: <function>,
     reduce: <function>,
     finalize: <function>,
     out: <output>,
     query: <document>,
     sort: <document>,
     limit: <number>,
     scope: <document>,
     jsMode: <boolean>,
     verbose: <boolean>,
     bypassDocumentValidation: <boolean>,
     collation: <document>
})

示例

示例集合person存储各省市居民的姓名、性别等记录

1
2
3
4
5
6
7
8
9
10
11
12
13
1.{
    "name" : "姓名1",
    "gender" : "男",
    "city" : "城市a",
    "province" : "省份A"
}
2.{
    "name" : "姓名2",
    "gender" : "女",
    "city" : "城市b",
    "province" : "省份B"
}
3...

现用mapReduce统计各省市人口性别比例：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
db.person.mapReduce(
    function(){//map
        var key = {province:this.province, city:this.city}
        var value = {total: 1}
        if (this.gender == '男'){
            value.male = 1
        }else if (this.gender == '女'){
            value.female = 1
        }else{
            value.unknown = 1
        }
        emit(key, value)
    },
    function(key, values){//reduce
        var value = {}//累计各性别的数量
        values.forEach(function(item){
            //item里可能是单个值，也可能已经是累计值
            for (var k in item){
                value[k] = (value[k]||0) + item[k]
            }
        })
        return value//reduce返回值不支持数组，若需多值请用对象格式
    },{
        query:{},//指定过滤源数据的查询条件
        sort:{province:1, city:1},//按key排序可减少reduce的次数，加快执行速度
        finalize: function(key, rValue){
            for (var k in rValue){
                if (k !== 'total'){//数量转为比例
                    rValue[k] = rValue[k]/rValue.total
                }
            }
            return rValue
        },
        out:{inline:true},
    }
).find()

执行结果如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
[
    {
        "_id" : {
            "province" : "省份A",
            "city" : "城市a"
        },
        "value" : {
            "total" : 40.0,
            "male" : 0.425,
            "female" : 0.575
        }
    },
    {
        "_id" : {
            "province" : "省份A",
            "city" : "城市b"
        },
        "value" : {
            "total" : 1.0,
            "male" : 1.0
        }
    },
    {
        "_id" : {
            "province" : "省份A",
            "city" : "城市c"
        },
        "value" : {
            "total" : 150.0,
            "male" : 0.526,
            "female" : 0.473
        }
    },
    ...
]

特别说明

  1. 当key对应的value只有一组时不会触发reduce。所以，map函数emit的value值的格式最好和reduce函数return值一致，避免最终的结果不一致。
  2. 同一个key可能会触发多次reduce，即reduce接收的values中有些item元素可能是已经reduced的，累积时要考虑这种情况（即values包含直接emit的value，也包含reduce返回的value）。
  3. reduce返回值不支持数组，若需多值请用对象格式。
  4. 按key排序可以减少reduce的次数，提高处理效率。
  5. out可以指定输出到一个collection集合，但这样会导致写操作；指定为{inline:true}时mongodb会用runReadCommand代替runCommand——if inline output is specified, we need to apply readPreference on the command as it could be run on a secondary.

JS的“缺陷”

JS从诞生到现在已经二十几年了，在其演进中难免引入了一些bug，且积习难改。理解这些缺陷有助于避开那些新手陷阱。

typeof相关

Null和Undefined

他们都是基本类型，也都只对应一个常量值（分别是null和undefined）。

1
2
3
4
5
6
//已知:
typeof undefined === 'undefined'
//你可能会以为:
typeof null === 'null'//但，不是这样
//真相是:
typeof null === 'object'//null是指空对象，空对象也是对象！

Object、Array、Function、RegExp、Date、Error、…

这些本质上都是对象类型。

1
2
3
4
5
6
7
//已知：
typeof {} === 'object'
typeof function(){} === 'function'
//你可能会以为:
typeof [] === 'array'//但，不是这样的
//真相是：
typeof [] === 'object'//数组也是对象，基本类型外的除了函数都是对象。

typeof一览表

类型 Type 值 typeof
布尔 Boolean true ‘boolean’
数值 Number 1 ‘number’
字符串 String ‘’ ‘string’
符号 Symbol Symbol() ‘symbol’
未定义 Undefined undefined ‘undefined’
空* Null null ‘object’
对象 Object {} ‘object’
下列都属于 对象的派生类型
数组 Array [] ‘object’
函数* Function function(){} ‘function’
类 Function class{} ‘function’

相等比较

宽松比较==

相等操作符比较两个值是否可能相等，即如果类型不同则进行可能的类型转换。

1
2
3
4
5
6
7
8
9
0==''//true
0=='0'//true
0==false//true
[1,2]=='1,2'//true
null==undefined//true
!null && !undefined//true
//注意：null和undefined既不等于true也不等于false
null==true || null==false//false
undefined==true || undefined==false//false

严格比较===

全等操作符比较两个值是否严格相等，即要求类型一样，也要求值相同。

1
2
3
4
5
6
null===null//true
undefined===undefined//true
null===undefined//false
0==='0'//false
0===-0//true
NaN===NaN//false?--这个比较有性格

同值相等Object.is()比较

Object.js()比较的结果同全等操作符，除了下面这两个相反：

1
2
Objecct.is(NaN,NaN)//true
Objecct.is(0,-0)//false