Abstract: With the recent exciting achievements of Google’s artificial intelligence system in the game of Go, deep reinforcement learning (DRL) has witnessed considerable development. DRL combines the abilities of sensing and making decisions provided by deep learning and reinforcement learning. Natural language processing (NLP) involves a large number of vocabularies or statements that have to be represented, and its subtasks, such as the dialogue system and machine translation, involve many decision problems that are difficult to model. Because of the aforementioned reasons, DRL can be appropriately applied to various NLP tasks such as named entity recognition, relation extraction, dialogue system, image caption, and machine translation. Further, DRL is helpful in improving the framework or the training pipeline of the aforementioned tasks, and notable achievements have been obtained. DRL is not an algorithm or a method but a paradigm. Many researchers fit plenty of NLP tasks in this paradigm and achieve better performance. Specifically, in text generation based on the reinforcement learning paradigm, the learning process that is used to produce a predicted sequence from the given source sequence can be considered to be the Markov decision process (MDP). In MDP, an agent interacts with the environment by receiving a sequence of observations and scaled rewards and subsequently produces the next action or word. This causes the text generation model to achieve decision-making ability, which can result in future success. Thus, the text generation task integrated with reinforcement learning is an attractive and promising research field. This study presented a comprehensive introduction and a systemic overview. First, we presented the basic methods in DRL and its variations. Then, we showed the main applications of DRL during the text generation task, trace the development of DRL, and summarized the merits and demerits associated with these applications. The final section enumerated some future research directions of DRL combined with NLP.
Abstract: Severe energy crisis and environmental pollution are the foremost problems in the world today. Electric vehicles have several advantages over traditional internal combustion engine-based vehicles, such as high energy efficiency and low emissions, which are effective in alleviating the energy crisis and environmental problems. However, the electric vehicles’ performance is greatly affected by temperature. An excessively high temperature during the charging and discharging process may accelerate the degradation rate of a battery cell and shorten its lifespan. In contrast, an excessively low temperature may reduce the battery’s efficiency and affect its discharge capacity. Air-conditioning systems in electric vehicles consume electricity to create a comfortable environment in the passenger compartment. However, excessive temperature of the motor drive will decrease its efficiency. Therefore, the battery, passenger compartment and motor drive system must be maintained at adequate temperatures to ensure the safety, comfort, and economy of the electric vehicles. Previous studies usually focused on a single thermal management system at a time, such as a battery thermal management system, air-conditioning systems in electric vehicles, and motor thermal management system. This means that the coupling relationships between the above-mentioned thermal management systems and the performance analysis of the integrated thermal management system at the vehicle level were not properly investigated. This study focused on the key issues in the construction of an integrated thermal management system for electric vehicles. Firstly, the heat generation models of the battery, passenger compartment, and motor drive system were summarized. Secondly, the existing thermal management methods for these three systems were systematically reviewed. Especially, the research status, operation control, and performance evaluation of the integrated thermal management system were especially analyzed. Finally, the deficiencies of the previous studies were summarized and the research prospects were proposed. It is pointed out that it is necessary to study the accurate heat generation models, develop the compact and efficient integrated thermal management system, and optimize the operation control of the integrated thermal management system under a comprehensive performance evaluation system in the near future.
Abstract: With the rapid progress of the automated highway system, the issue of platoon stability, which might significantly affect highway traffic characteristics, such as traffic efficiency, traffic capacity, and traffic safety, has attracted considerable attention. A string of vehicles equipped with adaptive cruise control (ACC) and moving longitudinally in an automated manner is regarded as an autonomous vehicle platooning system. During car following, the quality of the ride could be poor and rear-end collisions could occur, particularly if the spacing and velocity errors are amplified to some extent as they propagate upstream. Research on platoon stability has been the focus of significant interest. However, a method to coordinate multiple sub-objectives dynamically during autonomous vehicle platooning against multiple traffic scenarios has not yet been developed. In this study, a multi-objective ACC algorithm for cooperative adaptive cruise control (CACC) platooning based on vehicle-to-vehicle (V2V) real-time communication technology, which enabled the interconnection of vehicles within a limited range to share vehicle position and motion state information, was thus proposed. The quantization of homogeneous and heterogeneous platoon stability was analyzed on the basis of the Lyapunov stability theory. Furthermore, on the basis of the model predictive control framework, the coordination among various conflicting sub-objectives, such as driver-desired car-following response, rear-end safety, platoon stability, and platoon overall quality, was comprehensively considered. Then, by utilizing a quadratic cost function with linear multi-constraints, the design of the multi-objective CACC was transformed into the convex quadratic programming problem with multiple constraints. The comparative simulations show that the I/O constraints and slack relaxation of platoon control are strict, indicating that platoon stability is easily affected by certain factors, such as time gap, platoon size, sub-objective weight coefficient, transient traffic scenarios, and heterogeneous features. Thus, rear-end safety and platoon stability should be prioritized to guarantee the overall quality of the platoon.
Abstract: A distributed optimization problem is cooperatively solved by a network of agents, which have significant applications in many science and engineering fields, such as metallurgical engineering. For complex industrial processes with multiple-level characteristics, varying working conditions, and long processes, numerous optimization decision-making micro and macro control problems, such as product quality control, production planning, scheduling, and energy comprehensive deployment, are encountered. The theory and method of distributed optimization are keys to promoting the strategic decision-making of the integration of industrialization and new-generation industrial revolution. Their development enhances the ability to deal with large-scale and complex problems of big data, which have important practical value and economic benefits. In this study, consensus optimization with set constraints in multi-agent networks was explored. A distributed algorithm with a fixed step size was proposed on the basis of a primal-dual gradient scheme. Parameters such as step size affect the convergence of the algorithm. As such, convergence should be analyzed first, and appropriate parameters should be subsequently set in accordance with convergence conditions. Existing works have constructed different Lyapunov functions by exploiting the specific iteration scheme of this algorithm and analyzing convergence. Conversely, a convergence analysis paradigm based on a Lyapunov function was proposed in this study for general fixed step size iteration schemes, which were similar to the analysis method of Lyapunov convergence for general differential equations. A suitable Lyapunov function was constructed for the distributed gradient algorithm, and a parameter setting range was obtained in accordance with the convergence conditions. The proposed method avoids the tedious and complicated analysis of algorithm convergence and parameter assignment. The theory and method presented in this study also provide a framework and systematic demonstration method for other types of distributed algorithms and may be regarded as future directions of distributed optimization.
Abstract: Attention deficit hyperactivity disorder (ADHD) is one of the most common mental disorders during childhood, which lasts until adulthood in most cases. In recent years, ADHD classification based on functional magnetic resonance imaging (fMRI) data has become a research hotspot. Most existing classification algorithms reported in the literature assume that samples are balanced; however, ADHD data sets are usually imbalanced. Imbalanced data sets can cause the performance degradation of a classifier by imbalanced learning, which tends to overfocus on the majority class. In this study, we considered an imbalanced neuroimaging classification problem: classification of ADHD using resting state fMRI. We used the functional connection matrix of fMRI as the classification feature and proposed a multi-objective data classification scheme based on a support vector machine (SVM) to aid the diagnosis of ADHD. In this scheme, the imbalanced data classification problem is formulated as an SVM model with three objectives: maximizing the margin, minimizing the sum of positive errors, and minimizing the sum of negative errors. Accordingly, the positive and negative sample empirical errors can be separately handled. Then, the model is solved by a multi-objective optimization method, i.e., normal boundary intersection method. A set of representative classifiers are computed for selection by decision makers. The proposed scheme was tested and evaluated on five data sets from the ADHD-200 consortium and compared with traditional classification methods. Experimental results show that the proposed three-objective SVM classification scheme is better than traditional classification methods reported in the literature. It can effectively address the data imbalance problem from the algorithm level. This scheme can be used in the diagnosis of ADHD as well as other diseases, such as Alzheimer’s and Autism.
Abstract: 5G network technology can meet the high requirements of cyberspace development in the performance of communication platforms. Massive MIMO (multiple-input multiple-output) antenna array is one of the core technologies of 5G. The mutual coupling effect of massive MIMO antenna arrays will greatly reduce the Shannon capacity. In future 5G antenna systems, the biggest challenge is how to effectively eliminate the mutual coupling between the antennas in the array. The same antenna radiation pattern may correspond to many different forms of antenna. These different antennas have the same radiation characteristics but different scattering characteristics. To reduce the mutual coupling of large-scale antenna arrays, antenna elements with low scattering characteristics should be selected. To address the mutual coupling problem of large-scale array antennas, the scattering characteristics of antenna elements are studied. On the basis of the “invisible” minimum scattering antenna in the open-circuit state, the scattering matrix of the minimum scattering antenna connected in series with a quarter-wavelength transparent network is deduced. Findings indicate that this setup is the suitable minimum scattering antenna in the short-circuit state. The scattering formulas of a corrugated horn antenna under short-circuit, open-circuit, and matching loads are deduced by using a series network model. On the basis of these formulas, the calculation methods of excess scattering, associated scattering, and mismatched scattering components of the antenna are deduced. Scattering measurements of an X-band corrugated horn antenna under short-circuit, open-circuit, and matching loads are performed. According to minimum scattering antenna theory, the excess scattering, associated scattering, and mismatched scattering of the antenna are separated. Then, the maximum and minimum scattering of the corrugated horn antenna are calculated by using the separated scattering components; the calculated minimum scattering is much lower than the scattering when the antenna matches. Scattering measurements of corrugated horn antenna under preset loads are carried out by using a sliding short-circuit device to apply variable loads. The measured values are consistent with the calculated maximum and minimum scattering values, thereby verifying the correctness of the research on the scattering characteristics of the element antenna. Results show that, in the design of a large-scale array antenna, not only the radiation characteristics of the antenna but also the scattering characteristics of the antenna should be considered to reduce the mutual coupling effect of the antenna.
Abstract: Due to the popularity of intelligent mobile devices, malwares in the internet have seriously threatened the security of industrial control systems. Increasing number of malware attacks has become a major concern in the information security community. Currently, with the increase of malware variants in a wide range of application fields, some technical challenges must be addressed to detect malwares and achieve security protection in industrial control systems. Although many traditional solutions have been developed to provide effective ways of detecting malwares, some current approaches have their limitations in intelligently detecting and recognizing malwares, as more complex malwares exist. Given the success of machine learning methods and techniques in data analysis applications, some advanced algorithms can also be applied in the detection and analysis of complex malwares. To detect malwares and consider the advantages of machine learning algorithms, we developed a detection framework for malwares that threatens the network security of industrial control systems through the combination of an advanced machine learning algorithm, i.e., reinforcement learning. During the implementation process, according to the actual needs of malware behavior detection, key modules including feature extraction, policy, and classification networks were designed on the basis of the intelligent features of reinforcement learning algorithms in relation to sequence decision and dynamic feedback learning. Moreover, the training algorithms for the above key modules were presented while providing the detailed functional analysis and implementation framework. In the application experiments, after preprocessing the actual dataset of malwares, the developed method was tested and the satisfactory classification performance for malware was achieved that verified the efficiency and effectiveness of the reinforcement learning-based method. This method can provide an intelligent decision aid for general malware behavior detection.
Abstract: In recent years, increasing incidents of drone intrusion have occurred, and the drone collisions have become common. As a result, accidents may occur in densely populated areas. Therefore, drone monitoring is an important research topic in the field of security. Although many types of drone monitoring programs exist, most of them are costly and difficult to implement. To solve this problem, in the 5G context, this study proposed a method of using a city’s existing monitoring network to acquire data based on a deep learning algorithm for drone target detection, constructing a recognizable drone, and tracking the unmanned aerial vehicle. The method used the improved YOLOv3 (You only look once) model to detect the presence of drones in video frames. The YOLOv3 algorithm is the third generation version of the YOLO series, belonging to the one-stage target detection algorithm. This algorithm has significant advantages over the two-stage type of algorithm in speed. YOLOv3 outputs the position information of the drone in the video frame. According to the position information, the PID (Proportion integration differentiation) algorithm was used to adjust the center of the camera to track the drone. Then, the parameters of the plurality of cameras were used to calculate the actual coordinates of the drone, thereby realizing the positioning. We built the dataset by taking photos of the drone's flight, searching and downloading drone pictures from the Internet, and labeling the drones in the image by using the labelImg tool. The dataset was classified according to the number of rotors of the drone. In the experiment, the detection model was trained by the dataset classified by the number of rotors. The trained model can achieve 83.24% accuracy and 88.15% recall rate on the test set, and speed of 20 frames per second on the computer equipped with NVIDIA GTX 1060 for real-time tracking.
Abstract: As a document recorded by professional medical personnel, electronic medical records contain a large and important clinical resource. How to use a large amount of potential information in electronic medical records has become one of the major research directions. Chinese electronic medical records are knowledge-intensive, in which the data has considerable research value. However, they have more complex entities because of the language features of Chinese, and the composite entity is long. These sentences components in the text are missing. Moreover, the boundaries of clinical entities are often unclear. Labeling corpus is a job that requires a great deal of manpower because of the technical language used in a given text. Therefore, the recognition of Chinese clinical named entities is a hard problem. Considering these characteristics of Chinese electronic medical records, this paper proposed a double-layer annotation model that combined with a domain dictionary and conditional random field (CRF). A medical domain dictionary was constructed by statistical analysis method, and combined with CRF to mark two different granularity labeling operations. The manually constructed medical domain dictionary has extremely high accuracy for the recognition of registered words, and machine learning could automatically recognize unregistered words. This work integrated the two aspects based on these advantages. With the proposed method, diseases, symptoms, drugs, and operations could be recognized from Chinese electronic medical records. Using the test dataset, the Macro-P with 96.7%, the Macro-R with 97.7% and the Macro-F1 with 97.2% were obtained. The recognition performance of the proposed method was greatly improved compared with that of a single-layer model. The recognition effect of deep neural network with attention was also analyzed, which did not perform well due to the size of the domain dataset. The experimental results show the efficiency of the double-layer annotation model for the named entity recognition of Chinese electronic medical records.
Abstract: With the advent of intelligent manufacturing and big data, the Made in China 2025 Initiative and Industry 4.0 have been paying increasing attention to automation and intelligent industrial equipment. In the background of the present times, the complexity and intelligence of computer numerical control (CNC) machine tools have been continuously improved, and the types and descriptions of CNC machine tools’ faults have increased, presenting serious challenges to equipment maintenance and diagnosis of CNC machine tools. In order to provide guarantee for accurate fault diagnosis of CNC machine tools, and to prolong the service life of CNC machine tools, it is necessary to improve the performance of named entity recognition system. Accordingly, the named entity recognition in the equipment and faults field of CNC machine tools were studied, taking the historical examinations and repair records of CNC machine tools as the research object. After analyzing the characteristics of fault description in the historical examinations and repair records, a named entity recognition method was proposed based on the combination of bidirectional long short-term memory (BLSTM) and conditional random field with loop (L-CRF). The first step is to input a sentence and segment and label the input sentence. The annotation corpus is combined with the pre-trained generated word vector by using Skip-gram model in Word2vec, and the word vector is converted into a word vector sequence through the word embedding layer. In the second step, the word vector sequence is integrated into the BLSTM layer to learn long term dependency information. The final step is to input the sentence expression into the L-CRF layer to obtain the global optimal sequence. The experimental results show that the method is superior to other named entity recognition methods, which lays a solid foundation for the intelligent maintenance and the real-time diagnostic tasks of CNC machine tools.
Abstract: The Internet of Things (IoT) has become an essential supporting platform for the present and future cyber-enabled services. Cellular networks is considered as the main channel of the data access for IoT terminals distributed in the region of interest, and they have an irreplaceable value, especially in wide-area coverage. Thus, it has a significant application value to reduce the downlink transmit power consumption of base stations under the restrictions of the coverage requirements for the green communication in heterogeneous cellular networks. A gradient descent algorithm was proposed based on smooth approximation and root mean square propagation. The algorithm could minimize the total downlink power consumption of base stations while satisfying the IoT service coverage. First, the penalty function method was used to simplify such an optimization problem with complicated constraints to a new one with simple constraints. Then, the non-derivative objective function was transformed by an approximation method into a derivable form. We also presented the close-form of the gradient of the objective function with respect to both the azimuths of the antennas installed in the base stations and the downlink transmit power levels related to these antennas. Finally, the gradient descent algorithm with root mean square propagation was used to execute the optimization of the newly approximated but smoothed version of the original objective function. Simulation experiments were conducted, and the results show that the proposed algorithm can significantly reduce the total power consumption of the downlink radio frequency transmit under the restrictions of the coverage ratio requirements in the region of interest. Furthermore, not only is the convergence speed of the proposed algorithm very fast, but also the oscillation phenomenon that occurs during the iterative procedure steps of the optimization is greatly suppressed by the proposed algorithm compared with the meta-heuristic algorithms and ordinary gradient descent method.
Abstract: With the development of cloud computing technology, more individuals and organizations have chosen cloud services to store and maintain their data and reduce the burden on local storage and corresponding maintenance costs. However, although the cloud computing infrastructure is more powerful and reliable than personal computing devices, the cloud storage server is not completely trusted due to various internal and external threats; therefore, users need to regularly check whether their data stored in the cloud server are intact. Therefore, in recent years, researchers have proposed a variety of schemes for data integrity auditing in cloud storage. Among them, in a part of public auditing schemes for cloud storage based on homomorphic authenticators, random sampling of data blocks, and random masking techniques, users need to store and maintain a two-dimensional (2D) table related to the index information of data blocks in the file. When a user’s outsource data need to be frequently updated to avoid forgery attacks due to the similar index value of data block being reused, the design and maintenance of the 2D table become cumbersome. In this study, to solve the abovementioned problem, an index–stub table structure was first proposed, which is simple and easy to maintain. On the basis of this structure, a third-party auditor auditing scheme with a privacy-preserving property was proposed for cloud storage. This scheme can effectively support various remote dynamic operations for outsource data at the block level. Then, a formal security proof for data integrity guarantee provided by the scheme was given under the random oracle model. A formal security analysis was also given for the privacy-preserving property of the audit protocol. Finally, the performance of the scheme was theoretically analyzed and compared with relevant experiments. Results indicate that the scheme has high efficiency.
Abstract: Affected by complex international factors in recent years, terrorism events are increasingly rampant in many countries, thereby posing a great threat to the gloal community. In addition, with the widespread use of emerging technologies in military and commercial fields, terrorist organizations have begun to use emerging technologies to engage in destructive activities. As the Internet and information technology develop, terrorism has been rapidly spreading in cyberspace. Terrorist organizations have created terrorism websites, established multinational networks of terrorist organizations, released recruitment information and even conducted training activities through various mainstream websites with a worldwide reach. Compared with traditional terrorist activities, cyber terrorist activities have a greater degree of destructiveness. Cybercrime and cyber terrorism have become the most serious challenges for societies. Terrorist organizations take advantage of the Internet in rapid dissemination of extremism ideas, and develop a large number of terrorists and supporters around the world, especially in developed Western countries. Terrorist organizations even use the Internet and “dark net” networks to conduct terrorist training, and their activities are concealed. As a result, the "lone wolf" terrorist attacks in various countries have emerged in an endless stream, which is difficult to prevent. This study proposed a method of extracting entities and attributes of terrorist events based on semantic role analysis, and provided technical support for monitoring and predicting cyberspace terrorism activities. Firstly, a naive Bayesian text classification algorithm is used to identify terrorism events on the cleaned text corpus collected from the Anti-Terrorism Information Site of the Northwest University of Political Science and Law. The keyword extraction algorithm TF-IDF is adopted for constructing the terrorism vocabularies from the classified text corpus, combining natural language processing technology. Then, semantic role and syntactic dependency analyses are conducted to mine the attributive post-targeting relationship, the name//place name//organization, and the mediator-like relationship. Finally, regular expressions and constructed lexical terrorism-specific vocabularies are used to extract six entities and attributes (occurrence time, occurrence location, casualties, attack methods, weapon types and terrorist organizations) of terrorism event based on the four types of triad short texts. The F1 values of the six types of entity attribute extraction evaluation results exceeded 80% based on the experimental data of 4221 articles collected. Therefore, the method proposed has practical significance for maintaining social public safety because of the positive effect in monitoring and predicting cyberspace terrorism events.
Abstract: The operation of cranes and other large machinery threatens the safety of transmission lines. In order to solve this problem in the transmission scenario, the research from the aspects of data enhancement, network structure and the hyperparameters of the algorithm were performed. And a new end-to-end transmission line threat detection method based on TATLNet were proposed in this paper, which included the suspicious areas generation network VRGNet and threat discrimination network VTCNet. VRGNet and VTCNet share part of the convolution network for feature sharing and we used the model compression to compress the model volume and improved the detection efficiency. The method can realize accurate detection of large-scale machinery invading in the transmission scene from the perspective of computer vision and system engineering. To mend the insufficient training data, the data set was expanded by a combination of various data enhancement techniques. The sufficient experiments were carried out to explore the multiple hyperparameters of this method, and its optimal configuration was studied by synthesizing detection accuracy and inference speed. The research results are sufficient. With increase in the number of grids, the accuracy and recall first increase and then decrease, whereas, the detection efficiency decreases rapidly with increase in the number of grids. Considering the detection accuracy and reasoning speed, 9 × 9 is the optimal division strategy. With the increase in the input image resolution, the detection accuracy increases steadily and detection efficiency decreases gradually. To balance the detection accuracy and inference efficiency, 480 × 480 is selected as the final image input resolution. Experimental results and field deployment demonstrate that compared with other lightweight object detection algorithms, this method has better accuracy and efficiency in large-scale machinery invasion detection such as cranes in transmission fields, and meets the demands of practical applications.
Abstract: High-speed steel contains a large amount of carbides, the shape and distribution of which have an important influence on its quality. To improve the distribution of carbides in M2 high-speed steel, the temperature field and the shape of the metal pool during the mold-rotation process were investigated in detail using a numerical simulation. Moreover, the effect of the mold-rotation speed on the electroslag remelting process was investigated using a rotating bifilar electroslag remelting furnace under laboratory conditions. The morphology and distribution of carbides in an ESR ingot were observed using an SEM, and the composition of carbides was analyzed through an electrolytic extraction experiment. Results show that with increase in mold rotation speed, the high-temperature zone of the slag pool moves from the core to the edge. Moreover, the temperature distribution becomes uniform. The depth of the metal pool becomes shallow, and the thickness of the two-phase region decreases, which results in a short local solidification time and small secondary dendrite spacing. Correspondingly, with the increase in the mold rotation speed, the slag skin of ESR ingot becomes thin and more uniform than earlier. The cooling intensity of the mold on the ESR ingot is high, and the carbide network begins to break and become thin. The morphology of carbides changes from flake to fine rod. XRD analysis determines whether the mold rotates or not, carbides always comprise M2C, MC, and M6C. However, the content of M2C increases and the contents of MC and M6C decrease with the increase in mold-rotation speed. The main reason for the improvement in the carbide structure is that the mold rotation decreases the metal pool depth and two-phase zone thickness, which improves the solidification conditions and reduces the element segregation.
Monthly, started in 1955 Supervising institution：Ministry of Education Sponsoring Institution：University of Science and Technology Beijing Editorial office：Editorial Department of Chinese Journal of Engineering Publisher：Science Press Chairperson：Ren-shu Yang Editor-in-Chief：Ai-xiang Wu ISSN 2095-9389CN 2095-9389
